mirror of https://github.com/sourcegraph/sourcegraph.git synced 2026-02-06 17:31:43 +00:00

History

Chris Smith b0bb67b47c Update the default Sourcegraph-supplied LLM models (#64281 ) This PR sets the defaults for "Sourcegraph supplied LLM models". ## When will these "defaults" be used? These models will _only_ be used IFF the Sourcegraph instance is _explicitly_ using the newer "modelConfiguration" site configuration data. (And opts into using Sourcegraph-supplied LLM models.) If the Sourcegraph instance is using the older "completions" configuration blob, then _only_ the user-supplied models will be used. (Or, based on the specific defaults defined in the code for the completions provider.) ## What about Cody Free or Cody Pro? 😬 yeah, we're going to need to deal with that later. Currently Sourcegraph.com is _not_ using the newer "modelConfiguration" site configuration, and instead we have some hacks in the code to ignore the internal modelconfig. See this "super-shady hack": `e5178a6bc0/cmd/frontend/internal/httpapi/completions/get_model.go (L425-L455)` So we are just erring on the side of having Cody Free / Cody Pro "do whatever they do now", and this PR won't have any impact on that. We _do_ want Sourcegraph.com to only return this data, but there are a few things we need to get straightened out first. (e.g. Cody Gateway being ware of mrefs, and having Cody Clients no longer using `dotcom.ts` to hard-code Cody Pro LLM models.) ## What does this PR actually _do_? 1. It updates the code in `cmd/cody-gateway-config` so that it will produce a new "supported-models.json" file. 2. I then ran the tool manually, the output of which was then written to `internal/modelconfig/embedded/models.json`. 3. That's it. For any Sourcegraph releases after this PR gets merged, the "Sourcegraph supplied LLM models" will be the newer set defined in `models.json`. (i.e. having these new defaults, and including "fireworks::v1::starcoder".) ## Test plan ~~I tested things locally, and unfortunately it doesn't look like any clients are filtering based on the model capabilities. So "StarCoder" is showing up in the Cody Web UI, despite failing at runtime.~~ Update: This was a problem on my end. This isn't an issue. ## Changelog NA?		2024-08-06 10:26:56 -07:00
..
BUILD.bazel	feature(cody): Data types for encoding LLM Model config via Sourcegraph site config (#63533 )	2024-07-01 16:33:43 -07:00
dotcom_models.go	Update the default Sourcegraph-supplied LLM models (#64281 )	2024-08-06 10:26:56 -07:00
main.go	Update the default Sourcegraph-supplied LLM models (#64281 )	2024-08-06 10:26:56 -07:00
providers.go	Update the default Sourcegraph-supplied LLM models (#64281 )	2024-08-06 10:26:56 -07:00
README.md	feature(cody): Data types for encoding LLM Model config via Sourcegraph site config (#63533 )	2024-07-01 16:33:43 -07:00
utils.go	feature(cody): Data types for encoding LLM Model config via Sourcegraph site config (#63533 )	2024-07-01 16:33:43 -07:00

README.md

Cody Gateway Config

This tool generates the JSON ModelConfiguration data which describes the set of LLM models that Cody Gateway currently supports. This information is used by Sourcegraph instances so they can pick up on any new LLM models as they get released.

The output of this tool is a .json file which we check into this repo. It will be embedded in Go binaries as a simple way of deploying the configuration blob.

Usage

# First do a dry run, and confirm any changes are reflected in the output.
go run ./cmd/cody-gateway-config -dry-run=true

# Run the tool and overwrite the existing file in your repo.
go run ./cmd/cody-gateway-config \
    -dry-run=false \
    -confirm-overwrite=true \
    -output-file="./internal/modelconfig/embedded/models.json"

# Run the unit tests which among other things verifies that the static
# ModelConfiguration document is well-formed.
go test ./cmd/frontend/internal/modelconfig/...
go test ./internal/modelconfig/...

# And finally, rerun the `sg lint` command, to ensure it is formatted
# as expected.
sg lint --fix format