mirror of
https://github.com/sourcegraph/sourcegraph.git
synced 2026-02-06 19:21:50 +00:00
188424d5ae
This PR exposes a new HTTP endpoint from the `frontend` that will return
the LLM models currently supported by the Sourcegraph instance. (Or a
500 if Cody isn't enabled or not configured correctly.)
This endpoint will be used by Cody clients in order to fetch LLM models
dynamically, so that new LLM models can automatically be picked up as
they are available from Cody Gateway. The underlying data structures
will also make it easier for the Sourcegraph instance to configure
arbitrary LLM providers, customize model settings, etc.
```zsh
curl \
-H "Authorization: token ${SG_LOCALHOST_TOKEN}" \
https://sourcegraph.test:3443/.api/modelconfig/supported-models.json
```
## Overview
This PR is the latest (long) series of steps for enabling arbitrary LLM
model configuration in the Sourcegraph backend. This PR introduces three
new components into the `cmd/frontend/internal/modelconfig` package:
### `builder.go`, `builder_test.go`
The `builder struct` is the logic for how we generate the LLM model
configuration data based on the current site configuration, what is
embedded in the Sourcegraph binary, and later, what we obtain from Cody
Gateway.
This is essentially how we apply things like the "model allow/deny list"
or "provider/model overrides" that will later be available in the
Sourcegraph site configuration.
For now, some of this functionality is inert. As there is no way for you
to actually specify the type of configuration data that the `builder`
supports. (e.g. you cannot specify the "LLM model allow or deny list",
or fetch models from Cody Gateway.)
So realistically, with this PR the only thing it is doing is just
converting a (`internal/modelconfig/types`) `SiteModelConfiguration`
into a `ModelConfiguration` object. (Which was added in
https://github.com/sourcegraph/sourcegraph/pull/63533.)
### `init.go`, `service.go`
Next is the `Service interface`, which is a new global service that can
be used anywhere within the `frontend` binary to fetch the current LLM
configuration data.
This will be used later to allow the Completions API endpoints to look
up various provider and LLM configuration data.
`init.go` supplies an initialization function to the Sourcegraph
machinery, and then it goes about loading the LLM model configuration
data and exposing it in a singleton object. Part of this is also
registering a "site configuration watcher", so that we will
automatically pick up on any changes to the site configuration that may
impact the LLM model configuration.
### `httpapi.go`
Finally, we expose a `HTTPHandlers struct` that contains a single
`http.HandlerFunc` that will serve the HTTP response.
My intention is to later add more endpoints, such as a way for Site
admins to easily `curl` the full configuration information. Or surface
diagnostic-type information about available models.
## Next Steps
With this piece, we now expose a view of the Sourcegraph instance's
currently supported LLM models. However, the existing completions API
stilly reads the site configuration directly. (And does not use the
newer, and more flexible `modelconfig` package.)
So the next step to continue wiring this up is to refactor the
`frontend/internal/completions` and `internal/completions` packages to
rely on the "newer style" configuration data. (Which 🤞 is faithfully
reproducing the current site configuration data.)
The end result should be a no-op, but will allow us to no longer rely on
hard-coded LLM model names anywhere in the codebase. And instead we can
just rely on the model configuration data.
## Test plan
Added new unit tests. Verified things worked manually. And looked
through existing unit tests for other components to sanity check some of
the assumptions made in newer code.
## Changelog
Sourcegraph instances how expose an HTTP endpoint that authenticated
users can call to get a list of LLM models supported by the Sourcegraph
instance. In the future this will be used to allow Cody users to select
the LLM model dynamically, based on what is currently available.
|
||
|---|---|---|
| .. | ||
| BUILD.bazel | ||
| debug.go | ||
| service.go | ||
| shared.go | ||