Commit Graph

36603 Commits

Author SHA1 Message Date
Release Bot
850291aa58
[Backport 5.4.5099] chore(tooling): bump Go version to 1.22.4 (#63392)
Bump for @evict 

## Test plan

CI passes with no complaints

## Changelog

- Bumped version of Go used to build to 1.22.4
 <br> Backport bb178ba729 from #63124

Co-authored-by: Noah S-C <noah@sourcegraph.com>
2024-06-20 15:36:58 +01:00
sourcegraph-release-bot
3458e82e36
[Backport 5.4.5099] release: never use build number in image family (#63178)
the executor image and docker mirror image should now follow the
following naming convention:

Image family:
`sourcegraph-executors-[nightly|internal|&#39;&#39;]-&lt;MAJOR&gt;-&lt;MINOR&gt;`
Image name:
`sourcegraph-executor-[nightly|internal|&#39;&#39;]-&lt;MAJOR&gt;-&lt;MINOR&gt;-&lt;BUILD_NUMBER&gt;`

example:
Image family: `sourcegraph-executors-5-4`
Image name: `sourcegraph-executor-5-4-277666`

## What happens during releases and _not_ releases?
#### Nightly
**`nightly` suffix**
Image family:
`sourcegraph-executors-nightly-&lt;MAJOR&gt;-&lt;MINOR&gt;`
Image name:
`sourcegraph-executor-nightly-&lt;MAJOR&gt;-&lt;MINOR&gt;-&lt;BUILD_NUMBER&gt;`
#### Internal
**`internal` suffix**
Image family:
`sourcegraph-executors-internal-&lt;MAJOR&gt;-&lt;MINOR&gt;`
Image name:
`sourcegraph-executor-internal-&lt;MAJOR&gt;-&lt;MINOR&gt;-&lt;BUILD_NUMBER&gt;`
#### Public / Promote to public

** No suffix **

Image family: `sourcegraph-executors-&lt;MAJOR&gt;-&lt;MINOR&gt;`
Image name:
`sourcegraph-executor-&lt;MAJOR&gt;-&lt;MINOR&gt;-&lt;BUILD_NUMBER&gt;`

&gt;  [!IMPORTANT]
&gt; Should we keep the imagine name stable at
`sourcegraph-executor-&lt;MAJOR&gt;-&lt;MINOR&gt;-&lt;BUILD_NUMBER&gt;`
&gt; and only change the family name? 
&gt;
&gt; **Why?**
&gt;
&gt; The Image family dictates the collection of images and that changes
each major minor and or release phase so there is really no use in
changing the image name too, except at a glance you can see from the
name what image family it belongs to?
## Test plan




## Changelog




 <br> Backport 8bb0ab54eb from #63157

Co-authored-by: William Bezuidenhout <william.bezuidenhout@sourcegraph.com>
2024-06-10 09:29:06 +01:00
sourcegraph-release-bot
04e2234477
[Backport 5.4.5099] chore(release): remind releasers of tag pipeline (#63152)
Co-authored-by: Craig Furman <craig.furman@sourcegraph.com>
2024-06-07 10:58:58 +00:00
Craig Furman
2cfadb4826
[Backport 5.4.5099] fix(release): remove generate-notes from github r… (#63148)
Generate notes currently generates a changelog all the way from the tag
`app-2023` ... which means the content is more than what is allowed by
github - hard cap of 125000 characters.

(cherry picked from commit 817738a3ae)

Also backports https://github.com/sourcegraph/sourcegraph/pull/63119

Co-authored-by: William Bezuidenhout <william.bezuidenhout@sourcegraph.com>
2024-06-07 11:17:33 +01:00
sourcegraph-release-bot
fd1fe425c1
[Backport 5.4.5099] scim: Fix random suffix added to every user on resync (#63131)
In the current implementation, we check if there&#39;s a user in the
database already, and if so we add a random suffix to increase the
chances of the insert/update to succeed.

However, we didn&#39;t check if the user that already exists is _the
same user_. So for a first sync, usernames would look nice and tidy, but
then on a second sync from the SCIM provider, every user would get a
suffix.

This PR fixes that by adding a check for the user ID.

Test plan:

Added tests, verified SCIM sync doesn&#39;t modify usernames anymore
locally. <br> Backport 614375e538 from
#63122

Co-authored-by: Erik Seliger <erikseliger@me.com>
2024-06-06 17:40:27 +00:00
James Cotter
bcc4367f86
msp/deploy: add 'author' and 'commit_message' annotations (#63108)
Add 'author' and 'commit_message' annotations on release

## Test plan
CI
2024-06-05 11:43:02 -07:00
Beatrix
cff84a0a99
fix(gateway): update Google model names in allowed custom chat models (#63109)
Fix: Add missing `/` to provider name for google models in Cody Gateway
allowlist.

I was getting the following error from Cody Gateway:

```
event: error
data: {"error":"Sourcegraph Cody Gateway: unexpected status code 400: {\"error\":\"model \\\"google/\\\" is not allowed, allowed: [google/gemini-1.5-pro-latest, google/gemini-1.5-flash-latest]\"}\n"}
```

Changes:

- prepend "google/" to the Google model names to match the expected
format

<!-- 💡 To write a useful PR description, make sure that your description
covers:
- WHAT this PR is changing:
    - How was it PREVIOUSLY.
    - How it will be from NOW on.
- WHY this PR is needed.
- CONTEXT, i.e. to which initiative, project or RFC it belongs.

The structure of the description doesn't matter as much as covering
these points, so use
your best judgement based on your context.
Learn how to write good pull request description:
https://www.notion.so/sourcegraph/Write-a-good-pull-request-description-610a7fd3e613496eb76f450db5a49b6e?pvs=4
-->


## Test plan

<!-- All pull requests REQUIRE a test plan:
https://docs-legacy.sourcegraph.com/dev/background-information/testing_principles
-->

The issue doesn't affect instances that set `google` as the `provider`,
and their requests should work as expected.

This change only affect Cody Gateway users with the following setting:

```
"completions": {
    "provider": "sourcegraph"
    "chatModel": "google/gemini-1.5-pro-latest",
    "fastChatModel": "google/gemini-1.5-flash-latest",
  },
```

## Changelog

<!--
1. Ensure your pull request title is formatted as: $type($domain): $what
2. Add bullet list items for each additional detail you want to cover
(see example below)
3. You can edit this after the pull request was merged, as long as
release shipping it hasn't been promoted to the public.
4. For more information, please see this how-to
https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c?

Audience: TS/CSE > Customers > Teammates (in that order).

Cheat sheet: $type = chore|fix|feat $domain:
source|search|ci|release|plg|cody|local|...
-->

<!--
Example:

Title: fix(search): parse quotes with the appropriate context
Changelog section:

## Changelog

- When a quote is used with regexp pattern type, then ...
- Refactored underlying code.
-->
2024-06-05 11:20:13 -07:00
Robert Lin
b574300714
feat/enterprise: make per-subscription model allowlists no-op (#62912)
With #62911, per-enterprise-subscription model allowlists are no longer
respected, so we can safely update the UI to remove mentions of
allowlists, and also update our various allowlist-evaluation mechanisms
in `licensing` and GraphQL resolvers to just provide a wildcard
allowlist instead. It's not strictly required, but will make how the
model allowlists work more clearer/explicit.

Because we have a [planned migration for all this state to Enterprise
Portal](https://linear.app/sourcegraph/project/kr-enterprise-portal-manages-all-enterprise-subscriptions-12f1d5047bd2/overview),
we're not making any database changes.

Part of https://linear.app/sourcegraph/issue/CORE-135

### Context

In https://sourcegraph.slack.com/archives/C05SZB829D0/p1715638980052279
we shared a decision we landed on as part of #62263:

> Ignoring (then removing) per-subscription model allowlists: As part of
the API discussions, we've also surfaced some opportunities for
improvements - to make it easier to roll out new models to Enterprise,
we're not including per-subscription model allowlists in the new API,
and as part of the Cody Gateway migration (by end-of-June), we will
update Cody Gateway to stop enforcing per-subscription model allowlists.
Cody Gateway will still retain a Cody-Gateway-wide model allowlist.
[@chrsmith](https://sourcegraph.slack.com/team/U061QHKUBJ8) is working
on a broader design here and will have more to share on this later.

This means there is one less thing for us to migrate as part of
https://github.com/sourcegraph/sourcegraph/pull/62934, and avoids the
need to add an API field that will be removed shortly post-migration.

As part of this, rolling out new models to Enterprise customers no
longer require additional code/override changes.

## Test plan

Various tests pass.

Visual inspection of `sg start dotcom`:

- **Before:** <img width="947" alt="image"
src="https://github.com/sourcegraph/sourcegraph/assets/23356519/2dc0ab72-c77d-4c0e-a57e-4c336041da4e">
-
**After:**![image](https://github.com/sourcegraph/sourcegraph/assets/23356519/ab2e6d97-38c7-42df-9a5f-2448793db0cc)
2024-06-05 17:08:58 +00:00
Robert Lin
27211dea73
feat/msp: update handbook link in alerts dashboard, sort custom alerts first (#63089)
1. The dashboard link still points to the old `go/msp-ops/...` which no
longer work (CORE-105)
2. Alerts defined on top of the MSP defaults are probably of more
interest, so let's sort these in front of the others

## Test plan

Unit/golden tests
2024-06-05 09:09:22 -07:00
sourcegraph-buildkite
872e23f956
security: Auto-update package lockfiles for Sourcegraph base images (#63104)
Automatically generated PR to update package lockfiles for Sourcegraph
base images.

Built from Buildkite run
[#277149](https://buildkite.com/sourcegraph/sourcegraph/builds/277149).
## Test Plan
- CI build verifies image functionality

Co-authored-by: Buildkite <buildkite@sourcegraph.com>
2024-06-05 16:04:36 +00:00
Matthew Manela
8e804335ce
chore(graph): Add changelog entry for Python CTags fix (#63103)
Update changelog for 5.4 release to include the fix we did in
[scip-ctags](https://github.com/sourcegraph/sourcegraph/pull/62345)
2024-06-05 16:00:18 +00:00
Will Dollman
6fbb78abb0
Revert "security: Auto-update package lockfiles for Sourcegraph base images (#63067)" (#63102)
This reverts commit 3fc155dc4d.

<!-- 💡 To write a useful PR description, make sure that your description
covers:
- WHAT this PR is changing:
    - How was it PREVIOUSLY.
    - How it will be from NOW on.
- WHY this PR is needed.
- CONTEXT, i.e. to which initiative, project or RFC it belongs.

The structure of the description doesn't matter as much as covering
these points, so use
your best judgement based on your context.
Learn how to write good pull request description:
https://www.notion.so/sourcegraph/Write-a-good-pull-request-description-610a7fd3e613496eb76f450db5a49b6e?pvs=4
-->


## Test plan

- CI

<!-- All pull requests REQUIRE a test plan:
https://docs-legacy.sourcegraph.com/dev/background-information/testing_principles
-->


## Changelog

<!--
1. Ensure your pull request title is formatted as: $type($domain): $what
2. Add bullet list items for each additional detail you want to cover
(see example below)
3. You can edit this after the pull request was merged, as long as
release shipping it hasn't been promoted to the public.
4. For more information, please see this how-to
https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c?

Audience: TS/CSE > Customers > Teammates (in that order).

Cheat sheet: $type = chore|fix|feat $domain:
source|search|ci|release|plg|cody|local|...
-->

<!--
Example:

Title: fix(search): parse quotes with the appropriate context
Changelog section:

## Changelog

- When a quote is used with regexp pattern type, then ...
- Refactored underlying code.
-->
2024-06-05 15:42:44 +00:00
sourcegraph-buildkite
3fc155dc4d
security: Auto-update package lockfiles for Sourcegraph base images (#63067)
Automatically generated PR to update package lockfiles for Sourcegraph
base images.

Built from Buildkite run
[#277067](https://buildkite.com/sourcegraph/sourcegraph/builds/277067).
## Test Plan
- CI build verifies image functionality

Co-authored-by: Buildkite <buildkite@sourcegraph.com>
2024-06-05 16:24:13 +01:00
Peter Guy
d5a0415e7f
cleanup: fix typo: scretTokenKey --> secretTokenKey (#63101)
This one is straightforward: changing the spelling of a variable. 

As I poked around in the vscode extension looking for what's going on
with the authentication errors, the misspelling of `scretTokenKey`
became difficult to ignore. So I fixed it. :-D
 
## Test plan

1. Build and run locally.
```
cd client/vscode
pmpn run build
```
2. Launch extension in VSCode: open the `Run and Debug` sidebar view in
VS Code, then select `Launch VS Code Extension` from the dropdown menu.
Verify expected behavior.
2024-06-05 15:12:14 +00:00
Petri-Johan Last
5749db8faf
Initialise authProviders array in Go code (#63099)
When a Sourcegraph license is expired or revoked and the instance
reverts to free license, then `authz` (for permissions syncing) stops
working as well. This means that there are no AuthProviders set up and
the front-end code has `null` for these values.

The issue arrises from unitialised arrays in Go being marshalled as
`null` instead of `[]`. This change initialises the array explicitly,
instead of relying on the empty value.
2024-06-05 14:58:35 +00:00
Sahin Yort
1425097105
chore(bazel): upgrade to latest rules_oci 1.x (#63085)
Upgrades rules_oci from `1.4.3` to `1.7.6`, the latest 1.x release of
rules_oci before upgrading to rules_oci 2.x. Upgrading directly from
`1.4.3` to `2.0.0` is big a jump, because a lot has changed in between.

Signed-off-by: thesayyn <thesayyn@gmail.com>

## Test plan

I don't expect any breaking changes. Also, I am assuming the repo
already has a test coverage for containers built with rules_oci.


## Changelog
2024-06-05 14:35:37 +00:00
Noah S-C
4a93f29755
chore(bazel): enable rules_esbuild sandbox with object-inspect workaround (#61969)
Sandbox escapes be-gone

## Test plan

Tested in CI and locally with `bazel build //client/...` as well as a
lot of blood, sweat n tears tearing through failed sandboxes

## Changelog
2024-06-05 15:34:29 +01:00
Camden Cheek
d61368d2b5
Svelte: collapse path when too long (#63082)
This applies the `sizeToFit` action to the file header, allowing us to
collapse the header in multiple steps:
- First, collapse the actions down to icons
- Next, collapse elements one by one from the front of the path and put
them in an overflow dropdown
2024-06-05 07:54:33 -06:00
Rafał Gajdulewicz
08a1c6a6f6
Revert "Cody Gateway embeddings: powering with generated metadata" (#63098)
Reverts sourcegraph/sourcegraph#63000 - this makes Cody Gateway hang
(and fail to listen on 9992 port).
2024-06-05 14:56:03 +02:00
David Veszelovszki
23eed1d5ef
Fix Cody onboarding redirect problem (#63096) 2024-06-05 12:07:08 +00:00
Jan Hartman
4327bf8fc1
Cody Gateway embeddings: powering with generated metadata (#63000)
Connected to https://github.com/sourcegraph/bfg-private/pull/189 and
https://github.com/sourcegraph/cody/pull/4414.

We're introducing a hacky MVP to enable embeddings being powered by
metadata that's generated from code. This PR is the bare minimum to make
this work on CG. We plan to trigger metadata generation only if we're
using a new (fake) model (this comes in via a feature flag) and if the
request isn't a real-time query, but is a background indexing request.
The implementation is really hacky, but is also really minimal.

## Test plan
Testing locally through a feature flag.
2024-06-05 13:33:10 +02:00
Anton Sviridov
507668fefd
GRAPH-646: scip syntax java package namespace fix (#63048)
- Fix package declaration queries for Java in scip-syntax
- Fix symlink to precise SCIP data in integration tests
- Remove colored output from evaluation results
2024-06-05 10:51:11 +01:00
William Bezuidenhout
224b3e5830
bazel: rules_js rc3 and remove deprecated "exclude_declarations" option (#63095)
`exclude_declarations_from_npm_packages` is not an option anymore as
from the output
```
ERROR: @aspect_rules_js//npm:exclude_declarations_from_npm_packages :: Unrecognized option: @aspect_rules_js//npm:exclude_declarations_from_npm_packages
```

Also upgraded to `rc3` while diagnosing this.

Closes https://github.com/sourcegraph/devx-support/issues/1005
## Test plan
Tested locally + CI
```
sg bazel test //internal/appliance/reconciler:reconciler_test
INFO: Invocation ID: 70da4295-36f2-43a8-a71e-9b11ae489657
WARNING: Build option --modify_execution_info has changed, discarding analysis cache (this can be expensive, see https://bazel.build/advanced/performance/iteration-speed).
INFO: Analyzed target //internal/appliance/reconciler:reconciler_test (0 packages loaded, 17313 targets configured).
INFO: Found 1 test target...
Target //internal/appliance/reconciler:reconciler_test up-to-date:
  bazel-bin/internal/appliance/reconciler/reconciler_test_/reconciler_test
Aspect @@rules_rust//rust/private:rustfmt.bzl%rustfmt_aspect of //internal/appliance/reconciler:reconciler_test up-to-date (nothing to build)
Aspect @@rules_rust//rust/private:clippy.bzl%rust_clippy_aspect of //internal/appliance/reconciler:reconciler_test up-to-date (nothing to build)
INFO: Elapsed time: 1.210s, Critical Path: 0.11s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
//internal/appliance/reconciler:reconciler_test                 (cached) PASSED in 19.2s
```

## Changelog
- remove deprecated option `exclude_declarations_from_npm_packages` from
local.bazelrc
- update to rc3 of rules_js
2024-06-05 10:33:37 +02:00
William Bezuidenhout
605b2305eb
chore(sg): move registry list cmd to release list (#63094)
Follow up from https://github.com/sourcegraph/sourcegraph/pull/63079

## Test plan
Tested locally

## Changelog
2024-06-05 10:25:38 +02:00
William Bezuidenhout
e4eec6668a
feat(sg): respect the context when executing interrupt hooks (#63069)
During testing I found that sometimes some hooks would just hang and not
complete. In this PR we execute all hooks within a timeout context.
Ensuring we give _some_ time for hooks to execute but also making sure
we eventually exit if some hook is misbehaving.

Additional changes:
- Global timeout for all hook execution is 2 seconds
- We hard exit after 5 intterupts instead of 2
- Hooks are split into two groups: sequential and concurrent. As per
their names the hooks are executed differently depending how they were
registered.


## Test plan
Tested locally

```
^C⚠️ Interrupt received, executing hook groups for graceful shutdown...
⚠️ Executing 16 'cleanup' hooks for graceful shutdown...
[   repo-updater] INFO repo-updater.repo-updater.grpcserver grpcserver/grpcserver.go:76 Shutting down gRPC server
[   repo-updater] INFO sync_worker workerutil/worker.go:252 Shutting down dequeue loop {"name": "repo_sync_worker", "reason": ""}
worker stopped due to context error: context canceled
gitserver-1 stopped due to context error: context canceled
searcher stopped due to context error: context canceled
gitserver-0 stopped due to context error: context canceled
blobstore stopped due to context error: context canceled
symbols stopped due to context error: context canceled
caddy stopped due to context error: context canceled
repo-updater stopped due to context error: context canceled
embeddings stopped due to context error: context canceled
frontend stopped due to context error: context canceled
zoekt-index-0 stopped due to context error: context canceled
syntax-highlighter stopped due to context error: context canceled
zoekt-web-1 stopped due to context error: context canceled
web stopped due to context error: context canceled
zoekt-web-0 stopped due to context error: context canceled
⚠️ Executing 6 'general' hooks for for graceful shutdown...
 failed to run zoekt-index-1.
stderr:
INFO server zoekt-sourcegraph-indexserver/main.go:1017 removing tmp dir {"tmpRoot": "/Users/william/.sourcegraph/zoekt/index-1/.indexserver.tmp"}
2024/06/04 09:15:03 updating index 6 github.com/sourcegraph/sourcegraph@HEAD=e55003da894490122546f876452f651aae65bb55 reason=content-mismatch
INFO server zoekt-sourcegraph-indexserver/main.go:432 updated index {"repo": "github.com/sourcegraph/sourcegraph", "id": 6, "branches": ["HEAD=e55003da894490122546f876452f651aae65bb55"], "duration": "19.21403925s"}
```


## Changelog
- Hard exit sg when 5 intterupt hooks are received
- Respect the context while executing interrupt hooks to ensure we still
exit if some hook is misbehaving
2024-06-05 10:06:58 +02:00
Taras Yemets
231c31279d
PLG: use react-query for async state management (#62939) 2024-06-05 07:18:00 +00:00
Robert Lin
2630d5fc56
chore/msp/runtime: add args as trace attributes, clean up tracing code (#63087)
This is a simplified version of what we have for in-Sourcegraph db
connections as well. They're not directly compatible because we use
`pgx` hooks.

## Test plan

n/a
2024-06-04 17:30:32 -07:00
Beatrix
f2590cbb36
Cody Gateway: Add Gemini models to PLG and Enterprise users (#63053)
CLOSE https://github.com/sourcegraph/cody-issues/issues/211 &
https://github.com/sourcegraph/cody-issues/issues/412 &
https://github.com/sourcegraph/cody-issues/issues/412
UNBLOCK https://github.com/sourcegraph/cody/pull/4360

* Add support for Google Gemini AI models as chat completions provider
* Add new `google` package to handle Google Generative AI client
* Update `client.go` and `codygateway.go` to handle the new Google
provider
* Set default models for chat, fast chat, and completions when Google is
the configured provider
* Add gemini-pro to the allowed list

<!-- 💡 To write a useful PR description, make sure that your description
covers:
- WHAT this PR is changing:
    - How was it PREVIOUSLY.
    - How it will be from NOW on.
- WHY this PR is needed.
- CONTEXT, i.e. to which initiative, project or RFC it belongs.

The structure of the description doesn't matter as much as covering
these points, so use
your best judgement based on your context.
Learn how to write good pull request description:
https://www.notion.so/sourcegraph/Write-a-good-pull-request-description-610a7fd3e613496eb76f450db5a49b6e?pvs=4
-->


## Test plan

<!-- All pull requests REQUIRE a test plan:
https://docs-legacy.sourcegraph.com/dev/background-information/testing_principles
-->

For Enterprise instances using google as provider:

1. In your Soucegraph local instance's Site Config, add the following:

```
    "accessToken": "REDACTED",
    "chatModel": "gemini-1.5-pro-latest",
    "provider": "google",
```

Note: You can get the accessToken for Gemini API in 1Password.

2. After saving the site config with the above change, run the following
curl command:

```
curl 'https://sourcegraph.test:3443/.api/completions/stream' -i \
-X POST \
-H 'authorization: token $LOCAL_INSTANCE_TOKEN' \
--data-raw '{"messages":[{"speaker":"human","text":"Who are you?"}],"maxTokensToSample":30,"temperature":0,"stopSequences":[],"timeoutMs":5000,"stream":true,"model":"gemini-1.5-pro-latest"}'
```

3. Expected Output:

```
❯ curl 'https://sourcegraph.test:3443/.api/completions/stream' -i \
-X POST \
-H 'authorization: token <REDACTED>' \
--data-raw '{"messages":[{"speaker":"human","text":"Who are you?"}],"maxTokensToSample":30,"temperature":0,"stopSequences":[],"timeoutMs":5000,"stream":true,"model":"gemini-1.5-pro-latest"}'

HTTP/2 200
access-control-allow-credentials: true
access-control-allow-origin:
alt-svc: h3=":3443"; ma=2592000
cache-control: no-cache
content-type: text/event-stream
date: Tue, 04 Jun 2024 05:45:33 GMT
server: Caddy
server: Caddy
vary: Accept-Encoding, Authorization, Cookie, Authorization, X-Requested-With, Cookie
x-accel-buffering: no
x-content-type-options: nosniff
x-frame-options: DENY
x-powered-by: Express
x-trace: d4b1f02a3e2882a3d52331335d217b03
x-trace-span: 728ec33860d3b5e6
x-trace-url: https://sourcegraph.test:3443/-/debug/jaeger/trace/d4b1f02a3e2882a3d52331335d217b03
x-xss-protection: 1; mode=block

event: completion
data: {"completion":"I","stopReason":"STOP"}

event: completion
data: {"completion":"I am a large language model, trained by Google. \n\nThink of me as","stopReason":"STOP"}

event: completion
data: {"completion":"I am a large language model, trained by Google. \n\nThink of me as a computer program that can understand and generate human-like text.","stopReason":"MAX_TOKENS"}

event: done
data: {}
```

Verified locally:


![image](https://github.com/sourcegraph/sourcegraph/assets/68532117/2e6c914d-7a77-4484-b693-16bbc394518c)

#### Before

Cody Gateway returns `no client known for upstream provider google`

```sh
curl -X 'POST' -d '{"messages":[{"speaker":"human","text":"Who are you?"}],"maxTokensToSample":30,"temperature":0,"stopSequences":[],"timeoutMs":5000,"stream":true,"model":"google/gemini-1.5-pro-latest"}' -H 'Accept: application/json' -H 'Authorization: token $YOUR_DOTCOM_TOKEN' -H 'Content-Type: application/json' 'https://sourcegraph.com/.api/completions/stream'

event: error
data: {"error":"no client known for upstream provider google"}

event: done
data: {
```

## Changelog

<!--
1. Ensure your pull request title is formatted as: $type($domain): $what
2. Add bullet list items for each additional detail you want to cover
(see example below)
5. You can edit this after the pull request was merged, as long as
release shipping it hasn't been promoted to the public.
6. For more information, please see this how-to
https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c?

Audience: TS/CSE > Customers > Teammates (in that order).

Cheat sheet: $type = chore|fix|feat $domain:
source|search|ci|release|plg|cody|local|...
-->

<!--
Example:

Title: fix(search): parse quotes with the appropriate context
Changelog section:

## Changelog

- When a quote is used with regexp pattern type, then ...
- Refactored underlying code.
-->

Added support for Google as an LLM provider for Cody, with the following
models available through Cody Gateway: Gemini Pro (`gemini-pro-latest`),
Gemini 1.5 Flash (`gemini-1.5-flash-latest`), and Gemini 1.5 Pro
(`gemini-1.5-pro-latest`).
2024-06-04 23:46:36 +00:00
Robert Lin
f952ceb8da
feat/cody-gateway: use wildcard for enterprise allowlists (#62911)
This change makes Cody Gateway always apply a wildcard model allowlist,
irrespective of what the configured model allowlist is for an Enterprise
subscription is in dotcom (see #62909).

The next PR in the stack,
https://github.com/sourcegraph/sourcegraph/pull/62912, makes the GraphQL
queries return similar results, and removes model allowlists from the
subscription management UI.

Closes https://linear.app/sourcegraph/issue/CORE-135

### Context

In https://sourcegraph.slack.com/archives/C05SZB829D0/p1715638980052279
we shared a decision we landed on as part of #62263:

> Ignoring (then removing) per-subscription model allowlists: As part of
the API discussions, we've also surfaced some opportunities for
improvements - to make it easier to roll out new models to Enterprise,
we're not including per-subscription model allowlists in the new API,
and as part of the Cody Gateway migration (by end-of-June), we will
update Cody Gateway to stop enforcing per-subscription model allowlists.
Cody Gateway will still retain a Cody-Gateway-wide model allowlist.
[@chrsmith](https://sourcegraph.slack.com/team/U061QHKUBJ8) is working
on a broader design here and will have more to share on this later.

This means there is one less thing for us to migrate as part of
https://github.com/sourcegraph/sourcegraph/pull/62934, and avoids the
need to add an API field that will be removed shortly post-migration.

As part of this, rolling out new models to Enterprise customers no
longer require additional code/override changes.

## Test plan

Set up Cody Gateway locally as documented, then `sg start dotcom`. Set
up an enterprise subscription + license with a high seat count (for a
high quota), and force a Cody Gateway sync:

```
curl -v -H 'Authorization: bearer sekret' http://localhost:9992/-/actor/sync-all-sources
```

Verify we are using wildcard allowlist:

```sh
$ redis-cli -p 6379 get 'v2:product-subscriptions:v2:slk_...'
"{\"key\":\"slk_...\",\"id\":\"6ad033f4-c6da-43a9-95ef-f653bf59aaac\",\"name\":\"bobheadxi\",\"accessEnabled\":true,\"endpointAccess\":{\"/v1/attribution\":true},\"rateLimits\":{\"chat_completions\":{\"allowedModels\":[\"*\"],\"limit\":660,\"interval\":86400000000000,\"concurrentRequests\":330,\"concurrentRequestsInterval\":10000000000},\"code_completions\":{\"allowedModels\":[\"*\"],\"limit\":66000,\"interval\":86400000000000,\"concurrentRequests\":33000,\"concurrentRequestsInterval\":10000000000},\"embeddings\":{\"allowedModels\":[\"*\"],\"limit\":220000000,\"interval\":86400000000000,\"concurrentRequests\":110000000,\"concurrentRequestsInterval\":10000000000}},\"lastUpdated\":\"2024-05-24T20:28:58.283296Z\"}"
```

Using the local enterprise subscription's access token, we run the QA
test suite:

```sh
$ bazel test --runs_per_test=2 --test_output=all //cmd/cody-gateway/qa:qa_test --test_env=E2E_GATEWAY_ENDPOINT=http://localhost:9992 --test_env=E2E_GATEWAY_TOKEN=$TOKEN
INFO: Analyzed target //cmd/cody-gateway/qa:qa_test (0 packages loaded, 0 targets configured).
INFO: From Testing //cmd/cody-gateway/qa:qa_test (run 1 of 2):
==================== Test output for //cmd/cody-gateway/qa:qa_test (run 1 of 2):
PASS
================================================================================
INFO: From Testing //cmd/cody-gateway/qa:qa_test (run 2 of 2):
==================== Test output for //cmd/cody-gateway/qa:qa_test (run 2 of 2):
PASS
================================================================================
INFO: Found 1 test target...
Target //cmd/cody-gateway/qa:qa_test up-to-date:
  bazel-bin/cmd/cody-gateway/qa/qa_test_/qa_test
Aspect @@rules_rust//rust/private:clippy.bzl%rust_clippy_aspect of //cmd/cody-gateway/qa:qa_test up-to-date (nothing to build)
Aspect @@rules_rust//rust/private:rustfmt.bzl%rustfmt_aspect of //cmd/cody-gateway/qa:qa_test up-to-date (nothing to build)
INFO: Elapsed time: 13.653s, Critical Path: 13.38s
INFO: 7 processes: 1 internal, 6 darwin-sandbox.
INFO: Build completed successfully, 7 total actions
//cmd/cody-gateway/qa:qa_test                                            PASSED in 11.7s
  Stats over 2 runs: max = 11.7s, min = 11.7s, avg = 11.7s, dev = 0.0s

Executed 1 out of 1 test: 1 test passes.
```
2024-06-04 22:29:20 +00:00
Robert Lin
a3fe573b59
fix/msp: flatten custom alert promQL query for GCP (#63084)
The GCP monitoring alert configuration expects, for some reason, a
single-line PromQL query only, otherwise the threshold doesn't work. In
configuration, however, we may want to write a multi-line query, for
ease of readability. This change automatically flattens the PromQL query
into a single line and strips extra spaces.

Part of CORE-161

## Test plan

Unit tests
2024-06-04 14:37:51 -07:00
Erik Seliger
7d35662a59
gitserver: Fix some cases of RevNotFound in CommitLog (#63063)
There are so many different error messages this can return .. we seem to have missed a few that we need to map to RevisionNotFound, found in dotcom gitserver logs.

Test plan:

Added tests for the cases.
2024-06-04 23:01:02 +02:00
Erik Seliger
252abb56f8
repoupdater(scheduler): Prevent race condition of schedule preloading (#63086)
My local instance has few repos enough that this doesn't happen, but on larger instances this preloading fights with the new preloading.
They are both best effort, and are meant to achieve the same thing.
Thus, this one is not required anymore, and we can delete it, after we added another one in https://github.com/sourcegraph/sourcegraph/pull/62891.

Test plan:

Verified with sleeps and logs locally that repos are correctly upserted in the schedule now.
2024-06-04 22:55:10 +02:00
Erik Seliger
54472b70a0
gitserver(fs): Remove caching of cloned state (#63066)
I thought it might be nice to save on a few IOPS from the Stat calls to check if a repo is cloned by caching it for a bit. But turns out that the mutex locking and unlocking is more expensive than the stat operation.

We can come back here later and see if we can optimize it, we never had this in the past so this will be no regression.

Some background on mutex performance: https://stackoverflow.com/questions/57562606/why-does-sync-mutex-largely-drop-performance-when-goroutine-contention-is-more-t

Closes SRC-322

Test plan:

Without the cache, the tests still pass.
2024-06-04 22:54:11 +02:00
Camden Cheek
bf851d4598
Svelte: fix editor button styling (#63080)
The editor button would wrap before the div would overflow, causing weird wrapping. This fixes that and also makes it a proper button.
2024-06-04 12:49:40 -06:00
Erik Seliger
03c05e5fda
repo-updater: Hydrate schedule on startup (#62891)
Currently, when repo-updater restarts it loses all intel it collected over time. That causes a large flood of git fetch requests after it restarts. Every repo will be enqueued for an immediate update.

This PR fixes that by populating the scheduler with an initial delay per repo that is calculated with the same heuristic that the scheduler uses when it's fully warmed up.

This should avoid fetching git repos that are very stale (most likely the majority on instances with many repos).

Test plan:

Ran it locally, verified the scheduler state using the instrumentation tool for it, the schedule looks as expected and most repos aren't scheduled for the next 8h.
2024-06-04 19:00:23 +02:00
David Veszelovszki
b1cbbc82e5
Change JetBrains IDE statuses from Beta to Stable (#63073) 2024-06-04 09:55:34 -07:00
Robert Lin
c5c8db6205
fix/enterpriseportal: fix registration of connectRPC handler options (#63058) 2024-06-04 09:00:01 -07:00
William Bezuidenhout
8f3a9d5260
sg: add command to fetch versions from release registry (#63079)
added a commnad to list versions from the release registry
2024-06-04 17:42:47 +02:00
David Veszelovszki
b7cb870a0a
feature(plg): Use custom Stripe checkout (#62955)
* Bug fix: Stripe was called twice
* Split "individual" vs. "team" case for page title
* Use WhiteIcon for Cody logo
* Add customUI to CreateCheckoutSessionRequest
* Use custom Stripe checkout, matching Rob's designs
* Use debouncing
* Add "loading" states
* Redirect after successful payment (needs more fixes)
2024-06-04 17:01:30 +02:00
Camden Cheek
c6ff22a814
Svelte: make search header primary (#62991)
This implements the change to move the search bar back into the header, and the repo header down a level.
2024-06-04 14:23:26 +00:00
Beatrix
03f5866380
Cody Gateway: remove feature flag requirement for Gemini models (#63055)
Cody Gateway: add google/gemini-1.5-pro-latest and google/gemini-1.5-flash-latest to allowed models

This change adds the Google Gemini 1.5 Pro and Flash models to the list of allowed models for Cody Gateway chat completions by default without requiring "cody-pro-gemini-enabled". This allows Cody Pro users to access these models.

The changes were made in the following files:
- `cmd/frontend/internal/dotcom/productsubscription/codygateway_dotcom_user.go`
- `internal/licensing/codygateway.go`
- `internal/licensing/codygateway_test.go`
2024-06-04 07:15:35 -07:00
Camden Cheek
a5699594f2
Svelte: clean up file tree (#63009)
After the changes to the file tree layout in #62981, we can clean up the styles and DOM a bit. These changes are partially in prep for experimenting with alternative "current path" representations. Comments inline.
2024-06-04 07:43:40 -06:00
William Bezuidenhout
9bbfd25fc4
feat(sg: add list-build subcommand to ci (#63071)
* sg: add `list-build` subcommand to ci

Add command to list builds in various states on a pipeline

* bazel

remove trailing '...' from commit printing
2024-06-04 13:41:44 +02:00
Greg Magolan
2d3d918ffa
chore(bazel): upgrade to rules_js 2.0 RC (#63022)
Bumps to rules_js (and friends) to 2.0 RCs.

This brings in performance improvements for analysis phase since npm package depsets and now much smaller. It also adds support for pnpm v9 and allows for linking js_library targets as 1p deps instead of npm_package targets. See https://github.com/aspect-build/rules_js/issues/1671 for more details.

## Test plan

CI

## Changelog
2024-06-04 11:26:42 +00:00
William Bezuidenhout
1a7e1b9686
build-tracker: remove old links (#63065) 2024-06-04 12:03:58 +01:00
Varun Gandhi
d313441f8d
chore(codeintel): usagesForSymbol - make start/end non-optional for now (#63059)
We can add support for optional start/end later
in a backwards-compatible way. Making these non-optional
helps simplify the initial implementation.
2024-06-04 18:38:49 +08:00
Craig Furman
9847d67a7a
feat(appliance): deploy codeinsights-db (#63042)
* Add optional diff extra args to compare-helm

I wanted this to do `diff --side-by-side`, may as well commit it as a
feature.

* feat(appliance): deploy codeinsights-db
2024-06-04 10:23:29 +00:00
Varun Gandhi
3adcd25b76
chore: Simplify RequestState/PositionalRequestArgs o11y code (#63040) 2024-06-04 18:08:48 +08:00
Varun Gandhi
20c1c15608
schema: Document perf scaling for occurrences(...) (#62941)
Fixes GRAPH-634
2024-06-04 12:07:44 +02:00
William Bezuidenhout
a23d241c6d
chore(pnpm): add telemetry package (#63070)
pnpm: add telemetry package
2024-06-04 09:43:02 +00:00