artin/sourcegraph
1
0
Fork 0
mirror of https://github.com/sourcegraph/sourcegraph.git synced 2026-02-06 12:51:55 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat/cody-gateway: use wildcard for enterprise allowlists (#62911)

Browse Source 
This change makes Cody Gateway always apply a wildcard model allowlist,
irrespective of what the configured model allowlist is for an Enterprise
subscription is in dotcom (see #62909).

The next PR in the stack,
https://github.com/sourcegraph/sourcegraph/pull/62912, makes the GraphQL
queries return similar results, and removes model allowlists from the
subscription management UI.

Closes https://linear.app/sourcegraph/issue/CORE-135

### Context

In https://sourcegraph.slack.com/archives/C05SZB829D0/p1715638980052279
we shared a decision we landed on as part of #62263:

> Ignoring (then removing) per-subscription model allowlists: As part of
the API discussions, we've also surfaced some opportunities for
improvements - to make it easier to roll out new models to Enterprise,
we're not including per-subscription model allowlists in the new API,
and as part of the Cody Gateway migration (by end-of-June), we will
update Cody Gateway to stop enforcing per-subscription model allowlists.
Cody Gateway will still retain a Cody-Gateway-wide model allowlist.
[@chrsmith](https://sourcegraph.slack.com/team/U061QHKUBJ8) is working
on a broader design here and will have more to share on this later.

This means there is one less thing for us to migrate as part of
https://github.com/sourcegraph/sourcegraph/pull/62934, and avoids the
need to add an API field that will be removed shortly post-migration.

As part of this, rolling out new models to Enterprise customers no
longer require additional code/override changes.

## Test plan

Set up Cody Gateway locally as documented, then `sg start dotcom`. Set
up an enterprise subscription + license with a high seat count (for a
high quota), and force a Cody Gateway sync:

```
curl -v -H 'Authorization: bearer sekret' http://localhost:9992/-/actor/sync-all-sources
```

Verify we are using wildcard allowlist:

```sh
$ redis-cli -p 6379 get 'v2:product-subscriptions:v2:slk_...'
"{\"key\":\"slk_...\",\"id\":\"6ad033f4-c6da-43a9-95ef-f653bf59aaac\",\"name\":\"bobheadxi\",\"accessEnabled\":true,\"endpointAccess\":{\"/v1/attribution\":true},\"rateLimits\":{\"chat_completions\":{\"allowedModels\":[\"*\"],\"limit\":660,\"interval\":86400000000000,\"concurrentRequests\":330,\"concurrentRequestsInterval\":10000000000},\"code_completions\":{\"allowedModels\":[\"*\"],\"limit\":66000,\"interval\":86400000000000,\"concurrentRequests\":33000,\"concurrentRequestsInterval\":10000000000},\"embeddings\":{\"allowedModels\":[\"*\"],\"limit\":220000000,\"interval\":86400000000000,\"concurrentRequests\":110000000,\"concurrentRequestsInterval\":10000000000}},\"lastUpdated\":\"2024-05-24T20:28:58.283296Z\"}"
```

Using the local enterprise subscription's access token, we run the QA
test suite:

```sh
$ bazel test --runs_per_test=2 --test_output=all //cmd/cody-gateway/qa:qa_test --test_env=E2E_GATEWAY_ENDPOINT=http://localhost:9992 --test_env=E2E_GATEWAY_TOKEN=$TOKEN
INFO: Analyzed target //cmd/cody-gateway/qa:qa_test (0 packages loaded, 0 targets configured).
INFO: From Testing //cmd/cody-gateway/qa:qa_test (run 1 of 2):
==================== Test output for //cmd/cody-gateway/qa:qa_test (run 1 of 2):
PASS
================================================================================
INFO: From Testing //cmd/cody-gateway/qa:qa_test (run 2 of 2):
==================== Test output for //cmd/cody-gateway/qa:qa_test (run 2 of 2):
PASS
================================================================================
INFO: Found 1 test target...
Target //cmd/cody-gateway/qa:qa_test up-to-date:
  bazel-bin/cmd/cody-gateway/qa/qa_test_/qa_test
Aspect @@rules_rust//rust/private:clippy.bzl%rust_clippy_aspect of //cmd/cody-gateway/qa:qa_test up-to-date (nothing to build)
Aspect @@rules_rust//rust/private:rustfmt.bzl%rustfmt_aspect of //cmd/cody-gateway/qa:qa_test up-to-date (nothing to build)
INFO: Elapsed time: 13.653s, Critical Path: 13.38s
INFO: 7 processes: 1 internal, 6 darwin-sandbox.
INFO: Build completed successfully, 7 total actions
//cmd/cody-gateway/qa:qa_test                                            PASSED in 11.7s
  Stats over 2 runs: max = 11.7s, min = 11.7s, avg = 11.7s, dev = 0.0s

Executed 1 out of 1 test: 1 test passes.
```
This commit is contained in:
Robert Lin 2024-06-04 15:29:20 -07:00 committed by GitHub
parent a3fe573b59
commit f952ceb8da
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
5 changed files with 57 additions and 33 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
cmd/cody-gateway/internal/actor/productsubscription/productsubscription.go
View File

@ -10,19 +10,20 @@ import (
"github.com/Khan/genqlient/graphql"
"github.com/gregjones/httpcache"
"github.com/sourcegraph/log"
"github.com/sourcegraph/sourcegraph/internal/collections"
"github.com/vektah/gqlparser/v2/gqlerror"
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/trace"
"github.com/sourcegraph/sourcegraph/cmd/cody-gateway/internal/actor"
"github.com/sourcegraph/sourcegraph/cmd/cody-gateway/internal/dotcom"
"github.com/sourcegraph/sourcegraph/internal/codygateway"
"github.com/sourcegraph/sourcegraph/internal/collections"
"github.com/sourcegraph/sourcegraph/internal/license"
"github.com/sourcegraph/sourcegraph/internal/licensing"
"github.com/sourcegraph/sourcegraph/internal/productsubscription"
sgtrace "github.com/sourcegraph/sourcegraph/internal/trace"
"github.com/sourcegraph/sourcegraph/lib/errors"
"github.com/sourcegraph/sourcegraph/cmd/cody-gateway/internal/actor"
"github.com/sourcegraph/sourcegraph/cmd/cody-gateway/internal/dotcom"
)
// SourceVersion should be bumped whenever the format of any cached data in this
@ -298,7 +299,7 @@ func newActor(source *Source, token string, s dotcom.ProductSubscriptionState, i
a.RateLimits[codygateway.FeatureChatCompletions] = actor.NewRateLimitWithPercentageConcurrency(
int64(rl.Limit),
time.Duration(rl.IntervalSeconds)*time.Second,
rl.AllowedModels,
[]string{"*"}, // allow all models that are allowlisted by Cody Gateway
concurrencyConfig,
)
}
@ -307,7 +308,7 @@ func newActor(source *Source, token string, s dotcom.ProductSubscriptionState, i
a.RateLimits[codygateway.FeatureCodeCompletions] = actor.NewRateLimitWithPercentageConcurrency(
int64(rl.Limit),
time.Duration(rl.IntervalSeconds)*time.Second,
rl.AllowedModels,
[]string{"*"}, // allow all models that are allowlisted by Cody Gateway
concurrencyConfig,
)
}
@ -316,7 +317,7 @@ func newActor(source *Source, token string, s dotcom.ProductSubscriptionState, i
a.RateLimits[codygateway.FeatureEmbeddings] = actor.NewRateLimitWithPercentageConcurrency(
int64(rl.Limit),
time.Duration(rl.IntervalSeconds)*time.Second,
rl.AllowedModels,
[]string{"*"}, // allow all models that are allowlisted by Cody Gateway
// TODO: Once we split interactive and on-interactive, we want to apply
// stricter limits here than percentage based for this heavy endpoint.
concurrencyConfig,

1
cmd/cody-gateway/qa/BUILD.bazel
View File

@ -18,6 +18,7 @@ go_test(
"//internal/completions/client/fireworks",
"//lib/errors",
"@com_github_stretchr_testify//assert",
"@com_github_stretchr_testify//require",
],
)

20
cmd/cody-gateway/qa/README.md
View File

@ -1,19 +1,21 @@
# Basic E2E / smoke test suite for Cody Gateway
To run against local instance ($TOKEN is a `sgd_` token):
To run against local instance ($TOKEN is an Enterprise subscription's `slk_` token) - note that local config like `CODY_GATEWAY_ANTHROPIC_ACCESS_TOKEN` etc. must be set in `sg.config.overwrite.yaml` for this to work:
```
bazel test --test_output=all //cmd/cody-gateway/qa:qa_test --test_env=E2E_GATEWAY_ENDPOINT=http://localhost:9992 --test_env=E2E_GATEWAY_TOKEN=$TOKEN
```sh
bazel test --runs_per_test=2 --test_output=all //cmd/cody-gateway/qa:qa_test --test_env=E2E_GATEWAY_ENDPOINT=http://localhost:9992 --test_env=E2E_GATEWAY_TOKEN=$TOKEN
```
To run against dev instance using dotcom user ($TOKEN is a `sgd_` token):
To run against dev instance using dotcom user ($TOKEN is a user's `sgd_` token):
```
bazel test --test_output=all //cmd/cody-gateway/qa:qa_test --test_env=E2E_GATEWAY_ENDPOINT=https://cody-gateway.sgdev.org --test_env=E2E_GATEWAY_TOKEN=$TOKEN
```sh
bazel test --runs_per_test=2 --test_output=all //cmd/cody-gateway/qa:qa_test --test_env=E2E_GATEWAY_ENDPOINT=https://cody-gateway.sgdev.org --test_env=E2E_GATEWAY_TOKEN=$TOKEN
```
To run against prod using a dotcom user ($TOKEN is a `sgd_` token):
To run against prod using a dotcom user ($TOKEN is a user's `sgd_` token):
```sh
bazel test --runs_per_test=2 --test_output=all //cmd/cody-gateway/qa:qa_test --test_env=E2E_GATEWAY_ENDPOINT=https://cody-gateway.sourcegraph.com --test_env=E2E_GATEWAY_TOKEN=$TOKEN
```
bazel test --test_output=all //cmd/cody-gateway/qa:qa_test --test_env=E2E_GATEWAY_ENDPOINT=https://cody-gateway.sourcegraph.com --test_env=E2E_GATEWAY_TOKEN=$TOKEN
```
The `--runs_per_test=2` flag in snippet above ensures we don't hit a Bazel cache, and runs the test twice for good meausre.

21
cmd/cody-gateway/qa/e2e_test.go
View File

@ -11,10 +11,10 @@ import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/sourcegraph/sourcegraph/internal/completions/client/fireworks"
"github.com/stretchr/testify/require"
"github.com/sourcegraph/sourcegraph/internal/codygateway"
"github.com/sourcegraph/sourcegraph/internal/completions/client/fireworks"
"github.com/sourcegraph/sourcegraph/lib/errors"
)
@ -42,12 +42,12 @@ func Test_Completions(t *testing.T) {
if errors.Is(err, errNotImplemented) {
t.Skip(string(f), err)
}
assert.NoError(t, err)
require.NoError(t, err)
resp, err := http.DefaultClient.Do(req)
assert.NoError(t, err)
require.NoError(t, err)
body, err := io.ReadAll(resp.Body)
assert.NoError(t, err)
assert.Equal(t, resp.StatusCode, http.StatusOK, string(body))
require.NoError(t, err)
assert.Equal(t, http.StatusOK, resp.StatusCode, string(body))
if stream {
assert.Contains(t, resp.Header.Get("Content-Type"), "text/event-stream")
}
@ -90,7 +90,14 @@ func Test_Embeddings_OpenAI(t *testing.T) {
assert.NoError(t, err)
assert.Equal(t, 1, len(response.Embeddings))
assert.Equal(t, model.dimensions, len(response.Embeddings[0].Data))
assert.Equal(t, model.firstValue, response.Embeddings[0].Data[0])
// This can drift somewhat, round the comparison to a few decimal places
// to avoid diffs like:
//
// expected: -0.036106355
// actual : -0.03610423
assert.Equal(t,
fmt.Sprintf("%.4f", model.firstValue),
fmt.Sprintf("%.4f", response.Embeddings[0].Data[0]))
}
}

35
sg.config.yaml
View File

@ -299,17 +299,24 @@ commands:
go build -gcflags="$GCFLAGS" -o .bin/cody-gateway github.com/sourcegraph/sourcegraph/cmd/cody-gateway
checkBinary: .bin/cody-gateway
env:
CODY_GATEWAY_ANTHROPIC_ACCESS_TOKEN: foobar
# Set in override if you want to test local Cody Gateway: https://docs-legacy.sourcegraph.com/dev/how-to/cody_gateway
SRC_LOG_LEVEL: info
# Enables metrics in dev via debugserver
SRC_PROF_HTTP: '127.0.0.1:6098'
# Set in 'sg.config.overwrite.yaml' if you want to test local Cody Gateway:
# https://docs-legacy.sourcegraph.com/dev/how-to/cody_gateway
CODY_GATEWAY_DOTCOM_ACCESS_TOKEN: ''
CODY_GATEWAY_DOTCOM_API_URL: https://sourcegraph.test:3443/.api/graphql
CODY_GATEWAY_ALLOW_ANONYMOUS: true
CODY_GATEWAY_DIAGNOSTICS_SECRET: sekret
# Set in override if you want to test Embeddings with local Cody Gateway: http://go/embeddings-api-token-link
# Set in 'sg.config.overwrite.yaml' if you want to test upstream
# integrations from local Cody Gateway:
# Entitle: https://app.entitle.io/request?data=eyJkdXJhdGlvbiI6IjIxNjAwIiwianVzdGlmaWNhdGlvbiI6IldSSVRFIEpVU1RJRklDQVRJT04gSEVSRSIsInJvbGVJZHMiOlt7ImlkIjoiYjhmYTk2NzgtNDExZC00ZmU1LWE2NDYtMzY4Y2YzYzUwYjJlIiwidGhyb3VnaCI6ImI4ZmE5Njc4LTQxMWQtNGZlNS1hNjQ2LTM2OGNmM2M1MGIyZSIsInR5cGUiOiJyb2xlIn1dfQ%3D%3D
# GSM: https://console.cloud.google.com/security/secret-manager?project=cody-gateway-dev
CODY_GATEWAY_ANTHROPIC_ACCESS_TOKEN: sekret
CODY_GATEWAY_OPENAI_ACCESS_TOKEN: sekret
CODY_GATEWAY_FIREWORKS_ACCESS_TOKEN: sekret
CODY_GATEWAY_SOURCEGRAPH_EMBEDDINGS_API_TOKEN: sekret
SRC_LOG_LEVEL: info
# Enables metrics in dev via debugserver
SRC_PROF_HTTP: '127.0.0.1:6098'
CODY_GATEWAY_GOOGLE_ACCESS_TOKEN: sekret
watch:
- lib
- internal
@ -1025,17 +1032,23 @@ bazelCommands:
cody-gateway:
target: //cmd/cody-gateway
env:
CODY_GATEWAY_ANTHROPIC_ACCESS_TOKEN: foobar
SRC_LOG_LEVEL: info
# Enables metrics in dev via debugserver
SRC_PROF_HTTP: '127.0.0.1:6098'
# Set in override if you want to test local Cody Gateway: https://docs-legacy.sourcegraph.com/dev/how-to/cody_gateway
CODY_GATEWAY_DOTCOM_ACCESS_TOKEN: ''
CODY_GATEWAY_DOTCOM_API_URL: https://sourcegraph.test:3443/.api/graphql
CODY_GATEWAY_ALLOW_ANONYMOUS: true
CODY_GATEWAY_DIAGNOSTICS_SECRET: sekret
# Set in override if you want to test Embeddings with local Cody Gateway: http://go/embeddings-api-token-link
# Set in 'sg.config.overwrite.yaml' if you want to test upstream
# integrations from local Cody Gateway:
# Entitle: https://app.entitle.io/request?data=eyJkdXJhdGlvbiI6IjIxNjAwIiwianVzdGlmaWNhdGlvbiI6IldSSVRFIEpVU1RJRklDQVRJT04gSEVSRSIsInJvbGVJZHMiOlt7ImlkIjoiYjhmYTk2NzgtNDExZC00ZmU1LWE2NDYtMzY4Y2YzYzUwYjJlIiwidGhyb3VnaCI6ImI4ZmE5Njc4LTQxMWQtNGZlNS1hNjQ2LTM2OGNmM2M1MGIyZSIsInR5cGUiOiJyb2xlIn1dfQ%3D%3D
# GSM: https://console.cloud.google.com/security/secret-manager?project=cody-gateway-dev
CODY_GATEWAY_ANTHROPIC_ACCESS_TOKEN: sekret
CODY_GATEWAY_OPENAI_ACCESS_TOKEN: sekret
CODY_GATEWAY_FIREWORKS_ACCESS_TOKEN: sekret
CODY_GATEWAY_SOURCEGRAPH_EMBEDDINGS_API_TOKEN: sekret
SRC_LOG_LEVEL: info
# Enables metrics in dev via debugserver
SRC_PROF_HTTP: '127.0.0.1:6098'
CODY_GATEWAY_GOOGLE_ACCESS_TOKEN: sekret
docsite:
runTarget: //doc:serve
searcher: