Enables log sampling by default for Sourcegraph components that use the new internal logging library - the first 100 identical log entries per second will always be output, but thereafter only every 100th identical message will be output. It can be configured for each service using the environment variables SRC_LOG_SAMPLING_INITIAL and SRC_LOG_SAMPLING_THEREAFTER, and if SRC_LOG_SAMPLING_INITIAL is set to 0 or -1 the sampling will be disabled entirely.
We extend internal/tracer to support a new tracer type, "opentelemetry", that sends everything in OpenTelemetry format using the OpenTracing bridge. See https://github.com/sourcegraph/sourcegraph/pull/37984 for more details.
- 3a69966 indexserver: only log if someone else is running
- a6988a1 Export regexp optimisations
- ec21826 indexserver: allow specifying branches to index for sourcegraphFake
- b3aa3e8 go mod tidy
- 9208d95 merging: lock index dir during merge
includes
- 629b7ab indexserver: remove feature flag for explode
- dbb88ea indexserver: add IndexConcurrency for concurrent index jobs
- f331ab3 indexserver: introduce indexMutex
- b5426a7 read: inline delta-varint decoding in unmarshalDocumentSections
Empty output from git diff lead to the error message uneven pairs. This
is since bytes.Split on empty output returns "[][]byte{{}}".
Additionally added debug logging which would include the raw output. We
expect this error to not trigger, so this raw output would help
debugging.
Test Plan: Adjusting unit tests are a bit difficult at the moment, so I
relied on manual testing. I did a search on the default branch, but by
directly naming it so that searcher was used. After this commit it
works. For example
repo:^github\.com/sourcegraph/scip$@main UnspecifiedSymbolRole
Take two since including regexp optimization for some reason broke integration tests.
I may have done it in a bad way. The commit still exists for zoekt, we just won't apply
it for the regexes we create from sourcegraph just yet. So this is an unrevert but just
skips the unit test that was failing.
- 145fe82 Upgrade pcre2 package
- b61c934 indexData: experimental ngram map via binary search
- aacaf0c gomod: downgrade datadog
- 2eec411 gomod: bump ctags to avoid zombie procs
- b744e52 Convert regexp capture groups to non-capture groups
- a174ad6 fix inaccurate comment
- 7a91c48 Split `lineBounds` out into its own method
- f846f01 Create `newlines` type
- 5c5f94b improve descriptions of entries on /debug endpoint
- 25fa2fa builder.go: have incremental builds update latest commit date information
- 15dc905 indexserver: remove debug commands
- 6c563dc tests: fix alignment for number hints like "--0123"
- 0cafd7b all: stop using the deprecated package io/ioutil
- c16eb04 tests: replace ioutil.TempDir with testing.T.TempDir
- 6ca5909 go.mod: fix a cve in a transitive dependency to make Dependabot happy
- f2bb82f debug: add debug score for line matches
- a3e0a5f indexserver: support SG_PRIORITY file for FS fake
- fce3356 indexserver: always cleanup if index fails
- 794ad6c debug: new pages for List and listIndexed
- c85ca2f ranking: boost base filename matches
Test Plan: CI
- 145fe82 Upgrade pcre2 package
- b61c934 indexData: experimental ngram map via binary search
- aacaf0c gomod: downgrade datadog
- 2eec411 gomod: bump ctags to avoid zombie procs
- b744e52 Convert regexp capture groups to non-capture groups
- a174ad6 fix inaccurate comment
- 7a91c48 Split `lineBounds` out into its own method
- f846f01 Create `newlines` type
- 5c5f94b improve descriptions of entries on /debug endpoint
- 25fa2fa builder.go: have incremental builds update latest commit date information
- 15dc905 indexserver: remove debug commands
- 6c563dc tests: fix alignment for number hints like "--0123"
- 0cafd7b all: stop using the deprecated package io/ioutil
- c16eb04 tests: replace ioutil.TempDir with testing.T.TempDir
- 6ca5909 go.mod: fix a cve in a transitive dependency to make Dependabot happy
- f2bb82f debug: add debug score for line matches
- a3e0a5f indexserver: support SG_PRIORITY file for FS fake
- fce3356 indexserver: always cleanup if index fails
- 794ad6c debug: new pages for List and listIndexed
- c85ca2f ranking: boost base filename matches
Test Plan: CI
We removed all the uses of sqlx features during a refactor, and now its
existence is making other refactors difficult. This removes the sqlx
dependency and migrates its uses to `database/sql`.
* Add logic to store evaluated feature flags (when flagSet.GetBool|GetBoolOr called) in Redis to use when logging event logs
* Add new GraphQL API endpoints
* Add frontend feature flag utility defaultValue support
* Add tests
Co-authored-by: Erzhan Torokulov <erzhan.torokulov@gmail.com>
Co-authored-by: Camden Cheek <camden@ccheek.com>
The root cause of the go generate failures seems to be termenv.IsDarkBackground() hanging indefinitely in Buildkite. To work around this:
- Add ForceDarkBackground output option, set to true in Buildkite
- Refactor output capabilities detection to be lazy (if an override is set, do not do any detection at all)
- Change most usages of output.NewOutput to std.NewOutput for consistency and to avoid the problematic dark background detection
Reverts "ci: fix sg lint go hanging on generate failures (#36043)" (commit 8703224738)
* Add diff for a TypeScript file to trigger CI job
* Rename job
* Delete unnecessary TypeScript file after validating CI job
* Remove unused "LSIF Typed" code
This code got moved to a separate repo github.com/sourcegraph/scip.
This enables other tooling to take advantage of nice formatting, and also means we can integrate with some internal features, such as color toggling and terminal width detection.
With uncommitted changes, sg lint go is inaccurate because it reports all your code changes as generated changes. Maybe we can do some kind of stashing first, but that will affect other lints, so let's just do this for now.
Includes commits:
- 3bb00e7 indexserver: assert and check for ownership on index directory
- ec4be9d tests: make the code pass the go vet checks
- cdb5037 all: minor refactoring
Test Plan: CI
Includes the following changes:
- fe739e1 indexserver: debug list support -exclude_indexed
- 204b687 indexeserver: better logging for sourcegraphClient
- 3e8cba3 dockerfile: include curl
- 8a49c63 indexserver: seperate tmpfriend for debug vs main command
- a118209 indexserver: minor refactors around fetch
- 8efa934 ci: add shfmt
Test Plan: CI
Both gopkg.in/src-d/go-git.v4 and github.com/go-git/go-git/v5 are
imported in our project, and src-d/go-git is already deprecated. Let's
switch to go-git/go-git.
Signed-off-by: blanet <moweng.xx@alibaba-inc.com>
Co-authored-by: blanet <moweng.xx@alibaba-inc.com>
We have supported this in zoekt for many months, but the change was
never documented nor implemented in searcher. This PR finishes the task.
Test Plan: go test ./cmd/searcher/internal/search
