I was trying to understand exactly how a query I write is transalated
into a zoekt query. So this adds a command which allows me to easily try
stuff out and then see the s-expression output of the jobs plan.
Test Plan: A simple unit test is added to ensure this command doesn't
break.
The adds a new completions provider for use in the Sourcegraph App that
will send completions requests to sourcegraph.com. This will allow users
of Sourcegraph App to use Cody within App by using a dotcom account and
they will not need to configure App with an LLM key. App users that do
have their own API keys can apply a completions configuration if they
wish communicate directly with the LLM providers.
I chose not to add this provider as a configuration option because it
should only be used for Sourcegraph App and I wanted to avoid any
accidentally configurations using it.
This change will, by default, disable the HTML injection feature in
site-admin. Customers who want to customize their instance can add the
environment variable to add scripts or other HTML content. This will
make customers (the majority) who don't want this customization feature
protected against potential XSS attack.
## Test plan
Tested the changes on my local instance.
<!-- All pull requests REQUIRE a test plan:
https://docs.sourcegraph.com/dev/background-information/testing_principles
-->
I was playing around with the benchmark and was interested in the
throughput of the implementations. This is the numbers we seem to be
quoting in text which I assume previously we calculated manually.
Test Plan: go test -bench Search -run '^$'
This is to avoid the macos firewall popups. Also its just better to use
localhost for local dev.
Test Plan: Killed the running server then ensured the started server was
listening on localhost
Opening up as draft because I'm not sure about the changes we (@eseliger
and I) made here, but it fixed it locally.
## Test plan
- Manual testing of talking to Cody locally
When running Sourcegraph App the logic to determine if Cody is enabled
differs, because Cody can be enabled either by configuring the
completion settings or by connecting App to the users sourcegraph.com
account.
Closes#51345
Some more Cody completions tweaks:
1. We now include the suffix (the part of the prompt after the cursor)
into the knowledge base for more context. To avoid Cody repeating the
suffix without generting content, we cut away the first 5 lines of the
suffix though. This yielded better results in my limites testing.
If we didn't do that, Cody would just repeat the suffix in roughly two
out of three suggestions:
<img width="1609" alt="Screenshot 2023-05-03 at 15 25 05"
src="https://user-images.githubusercontent.com/458591/235961816-223a7053-f666-471a-8ec9-2d1f4f898ddc.png">
2. Better cut-off for multiline suggestions: We now compare the suffix
with the completion until we find 5 matching lines in the suffix. This
way, we can cut away content that is already in the suffix pretty
effectively, leading to smaller suggestion windows. E.g.:
<img width="1663" alt="Screenshot 2023-05-03 at 17 22 49"
src="https://user-images.githubusercontent.com/458591/235962380-f301f8b1-a2af-45e6-9eee-c66aeb9770af.png">
(Note: for this prompt I removed the highlighted part first. All three
suggestions are very relevant already!)
3. Removed the debug mode specialcasing for the multiline suggestions
window. It was tripping me off and I think if we do broader dogfooding
it might confuse others as well.
## Test plan
See screenshot above. Lots of back and forth 😬
<!-- All pull requests REQUIRE a test plan:
https://docs.sourcegraph.com/dev/background-information/testing_principles
-->
@valerybugakov I've noticed I missed this update in my recent PR to
update it.
Still I'm getting this locally when running things like `yarn run
format`:
```
~/work/sourcegraph U jh/doc/update-pnpm $ yarn run format
yarn run v1.22.17
warning @: The engine "pnpm" appears to be invalid.
```
Do you know how to fix this?
## Test plan
<!-- All pull requests REQUIRE a test plan:
https://docs.sourcegraph.com/dev/background-information/testing_principles
-->
CI
After much wrestling with Terraform and GCP, Redis and required
networking should now be up for the LLM-proxy. This adds a healthcheck
that pings it to make sure we are set up correctly.
The connection string is set to `REDIS_ENDPOINT` so the `redispool`
package should be available for use out-of-the-box.
## Test plan
`sg start llm-proxy`
```
$ curl localhost:9992/healthz
healthz: ok%
```
This pull request is part of the Sourcegraph 5.0.3 release.
* [Release batch
change](https://k8s.sgdev.org/organizations/sourcegraph/batch-changes/release-sourcegraph-5.0.3)
* [Tracking
issue](https://github.com/sourcegraph/sourcegraph/issues/51302)
### Test plan
CI checks in this repository should pass, and a manual review should
confirm if the generated changes are correct.
### ⚠️ Additional changes required
These steps must be completed before this PR can be merged, unless
otherwise stated. Push any required changes directly to this PR branch.
- [x] Ensure all other pull requests in the batch change have been
merged
- [x] Run `pnpm run release release:finalize` to generate the tags
required. CI will not pass until this command is run.
- [x] Re-run the build on this branch (using either the command `sg ci
build` or the Buildkite UI) and merge when the build passes.
This fixes an issue where private repos (repos with permissions enabled)
were skipped during indexing ownership recent contributors. The indexer
needs access to see all repos and paths, which also implies we are going
to have to filter these out at query time.
## Test plan
I tested this using explicit permissions API, and included an automated
test for the same.
<!-- All pull requests REQUIRE a test plan:
https://docs.sourcegraph.com/dev/background-information/testing_principles
-->
This reverts commit af29d8e634.
The failure was happening because external service instructions were
created statically instead of being rendered when needed, so the correct
Link component was not yet set. The second commit in this PR addresses
the issue and converts all external service instructions into function
components that use the right Link components set at application start.
## Test plan
Verify main-dry-run passes
In the Tauri app, removes the overlay titlebar and moves the
back/forward buttons into the global navbar. This makes it so that the
back/forward navigation is not dependent on OS support for overlay
titlebar and that we don't need to deal with window decorations in the
overlay that may vary based on platform.
## Test plan
## Test plan
Verify by running locally and checking ping structure.
Verify that payload is properly formed and submitted to pubsub topic.
<!-- All pull requests REQUIRE a test plan:
https://docs.sourcegraph.com/dev/background-information/testing_principles
-->
---------
Co-authored-by: Joe Chen <joe@sourcegraph.com>
Issue: https://github.com/sourcegraph/sourcegraph/issues/50506
The issue manifests for a structural search for any revisions specified
in the query that have both of the following:
1) the revision is a commit and is the latest commit of an indexed
branch in Zoekt
2) the revision is not the branch name indexed in Zoekt (i.e. it is not
a revision specified in `experimentalFeatures.search.index.branches`)
The following fix works for all cases where the indexed branch to query
is `HEAD`. Therefore this will resolve the code insights use case since
code insights repo filter does not accept revisions.
The fix in this PR is intended to be something we can ship quickly and
therefore I hoped to avoid modifying existing logic as much as possible.
For this reason we keep the `SearchJob` arguments and logic unchanged
but just modify the repo revisions passed in.
Note that there is still a remaining issue where if we specify two
revisions for our query and both of those revisions match both criteria
described above, then we do not have enough information to determine
which branch to use since we'll have two branches and neither of them
have a map to their latest commit. To resolve this I believe we'd have
to modify
[`PartitionRepos()`](https://sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/internal/search/zoekt/indexed_search.go?L177:6)
and/or
[`IndexedRepoRevs`](https://sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/internal/search/zoekt/indexed_search.go?L38)
which would be a bit more involved so I am planning to spin off to a
separate ticket.
## Test plan
<!-- All pull requests REQUIRE a test plan:
https://docs.sourcegraph.com/dev/background-information/testing_principles
-->
New method has test cases added
This adds additional checks to CodyEnabled to only enable Cody when
completions are configured. Currently if completions are not configured
or are configured but disabled Cody still appears in the UI but all
requests will fail.
This PR hooks up the recent contributors indexer with the background
job, effectively enabling indexing once the feature flag is switched on.
## Test plan
You can start Sourcegraph and let the jobs auto-index, or you can
manually control the queue through the `own_background_jobs` table.
Here is some example output auto-indexing the local dev repos:
```
[worker] INFO worker.own-repo-indexing-queue.own.background.index.scheduler.recent-contributors background/scheduler.go:84 Scheduling repo indexes for own job {"job-name": "recent-contributors"}
[worker] INFO worker.own-repo-indexing-queue.own.background.index.scheduler.recent-contributors background/scheduler.go:96 Own index job scheduled {"job-name": "recent-contributors", "row-count": 11}
[worker] INFO worker.own-repo-indexing-queue workerutil/worker.go:326 Dequeued record for processing {"TraceId": "1e151b9be9a6cb308ab77719c6ad54c0", "SpanId": "e738c4f0d1af8d29", "name": "own_background_worker", "id": 123}
[worker] INFO worker.own-repo-indexing-queue workerutil/worker.go:326 Dequeued record for processing {"name": "own_background_worker", "id": 124}
[worker] INFO worker.own-repo-indexing-queue workerutil/worker.go:326 Dequeued record for processing {"TraceId": "4a857e335aa28ba2717c951cfff07ba5", "SpanId": "04e3ff4d85cb72bc", "name": "own_background_worker", "id": 125}
[worker] INFO worker.own-repo-indexing-queue workerutil/worker.go:326 Dequeued record for processing {"name": "own_background_worker", "id": 126}
[worker] INFO worker.own-repo-indexing-queue workerutil/worker.go:326 Dequeued record for processing {"TraceId": "e1b51f47de0dcbfdebd14bfc346f60a2", "SpanId": "0384937856ac05a4", "name": "own_background_worker", "id": 127}
[worker] INFO worker.own-repo-indexing-queue.Handle background/recent_contributors.go:67 commits inserted {"TraceId": "e1b51f47de0dcbfdebd14bfc346f60a2", "SpanId": "4c39331206218a30", "handle": {"count": 0, "repo_id": 6}}
[worker] INFO worker.own-repo-indexing-queue.Handle background/recent_contributors.go:67 commits inserted {"TraceId": "4a857e335aa28ba2717c951cfff07ba5", "SpanId": "f7bbf2aa07a3aef2", "handle": {"count": 0, "repo_id": 5}}
[worker] INFO worker.own-repo-indexing-queue workerutil/worker.go:326 Dequeued record for processing {"name": "own_background_worker", "id": 128}
[worker] INFO worker.own-repo-indexing-queue workerutil/worker.go:326 Dequeued record for processing {"name": "own_background_worker", "id": 129}
[worker] INFO worker.own-repo-indexing-queue.Handle background/recent_contributors.go:67 commits inserted {"handle": {"count": 0, "repo_id": 4}}
[worker] INFO worker.own-repo-indexing-queue workerutil/worker.go:326 Dequeued record for processing {"name": "own_background_worker", "id": 130}
[worker] INFO worker.own-repo-indexing-queue.Handle background/recent_contributors.go:67 commits inserted {"handle": {"count": 0, "repo_id": 8}}
[worker] INFO worker.own-repo-indexing-queue.Handle background/recent_contributors.go:67 commits inserted {"handle": {"count": 1, "repo_id": 1}}
[worker] INFO worker.own-repo-indexing-queue workerutil/worker.go:326 Dequeued record for processing {"name": "own_background_worker", "id": 131}
[worker] INFO worker.own-repo-indexing-queue workerutil/worker.go:326 Dequeued record for processing {"name": "own_background_worker", "id": 132}
[worker] INFO worker.own-repo-indexing-queue.Handle background/recent_contributors.go:67 commits inserted {"TraceId": "1e151b9be9a6cb308ab77719c6ad54c0", "SpanId": "b3615fd2c3989b98", "handle": {"count": 3, "repo_id": 11}}
[worker] INFO worker.own-repo-indexing-queue.Handle background/recent_contributors.go:67 commits inserted {"handle": {"count": 0, "repo_id": 9}}
```
<!-- All pull requests REQUIRE a test plan:
https://docs.sourcegraph.com/dev/background-information/testing_principles
-->
---------
Co-authored-by: Cezary Bartoszuk <cezarybartoszuk@gmail.com>
Co-authored-by: Alex Ostrikov <alex.ostrikov@sourcegraph.com>
Allows for values to the `-version` flag on the `drift` command to also
include version specifiers similar to the `up/upgrade` commands, instead
of a hard requirement on them being git resolvables (aka we use `v`
prefixes in the git repo, so v5.0.2 had to be specified, but now the `v`
can be excluded)
## Test plan
Tested by @DaedalusG on a call, see results below
---------
Co-authored-by: DaedalusG <warrenbruceg@gmail.com>
This is just running "bazel run :gazelle-update-repos". I added a keep
for the dependencies it removed since according to JH we still need them
around for backwards compatability tests.
Test Plan: bazel test //enterprise/cmd/frontend/internal/auth/saml/...
Allows formatting everything with `nix fmt`
[nil](https://github.com/oxalica/nil) appears more maintained and higher
quality than rnix-lsp
## Test plan
N/A, nix stuff
This PR is autogenerated to update to
sourcegraph/zoekt@f818d968dd
## Test Plan
Testing is done on the zoekt repo. All checks on this PR are sufficient.
---------
Co-authored-by: keegancsmith <keegancsmith@users.noreply.github.com>
Co-authored-by: Keegan Carruthers-Smith <keegan.csmith@gmail.com>
