mirror of
https://github.com/sourcegraph/sourcegraph.git
synced 2026-02-06 19:51:50 +00:00
[Cody GA Docs] Improve and Update Cody Gateway docs (#58175)
* Improve and Update Cody Gateway docs * Update doc/cody/core-concepts/cody-gateway.md Co-authored-by: Robert Lin <robert@bobheadxi.dev> * Add feedback * Add tweaks * Update doc/cody/core-concepts/cody-gateway.md Co-authored-by: Robert Lin <robert@bobheadxi.dev> * Fix broken link --------- Co-authored-by: Robert Lin <robert@bobheadxi.dev>
This commit is contained in:
Maedah Batool
GitHub
parent
c80f9be8f6
commit
0402f2ce1d
@ -412,3 +412,10 @@
|
||||
/cody/explanations/cody_gateway /cody/core-concepts/cody_gateway 308
|
||||
/cody/explanations/code_graph_context /cody/core-concepts/code-graph 308
|
||||
/cody/core-concepts/cody_clients /cody/overview#getting-started 308
|
||||
|
||||
# Cody Gateway redirects
|
||||
/cody/core-concepts/cody_gateway /cody/core-concepts/cody-gateway 308
|
||||
/cody/core-concepts/cody_gateway#using-cody-gateway-in-sourcegraph-enterprise /cody/core-concepts/cody-gateway#using-cody-gateway-in-sourcegraph-enterprise 308
|
||||
/cody/core-concepts/cody_gateway#configuring-custom-models /cody/core-concepts/cody-gateway#configuring-custom-models 308
|
||||
/cody/core-concepts/cody_gateway#rate-limits-and-quotas /cody/core-concepts/cody-gateway#rate-limits-and-quotas 308
|
||||
/cody/core-concepts/cody_gateway#privacy-and-security /cody/core-concepts/cody-gateway#privacy-and-security 308
|
||||
|
||||
75
doc/cody/core-concepts/cody-gateway.md
Normal file
75
doc/cody/core-concepts/cody-gateway.md
Normal file
@ -0,0 +1,75 @@
|
||||
# Sourcegraph Cody Gateway
|
||||
|
||||
<p class="subtitle">Learn how Cody Gateway powers the default Sourcegraph provider for completions and embeddings to facilitate Cody features.</p>
|
||||
|
||||
<aside class="badge">
|
||||
<p> <span style="margin-right:0.25rem;" class="badge badge-note">Sourcegraph 5.1+</span> Cody Gateway is supported for Sourcegraph Enterprise customers on version 5.1 or more. </p>
|
||||
</aside>
|
||||
|
||||
Sourcegraph Cody Gateway powers the default `"provider": "sourcegraph"`, Cody completions and embeddings for Sourcegraph Enterprise users. It supports a variety of upstream LLM providers, such as [Anthropic](https://www.anthropic.com/) and [OpenAI](https://openai.com/), with rate limits, quotas, and model availability tied to your Sourcegraph Enterprise subscription.
|
||||
|
||||
Code snippets are sent to these third-party LLM providers when you use the Cody extension or enable embeddings. Make sure you review the [Cody usage and privacy notice](https://about.sourcegraph.com/terms/cody-notice).
|
||||
|
||||
## Using Cody Gateway in Sourcegraph Enterprise
|
||||
|
||||
To enable completions and embeddings provided by Cody Gateway on your Sourcegraph Enterprise instance, make sure your license key is set, and Cody is enabled in your [site configuration](../../admin/config/site_config.md):
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"licenseKey": "<...>",
|
||||
"cody.enabled": true,
|
||||
}
|
||||
```
|
||||
|
||||
After adding the license key, the default configuration and authentication will be automatically applied.
|
||||
|
||||
For more details about configuring Cody, read the following resources:
|
||||
|
||||
- [Enabling Cody for Sourcegraph Enterprise](./../overview/enable-cody-enterprise.md)
|
||||
- [Code Graph Context: Embeddings](./code-graph.md#embeddings)
|
||||
|
||||
Cody Gateway is hosted at `cody-gateway.sourcegraph.com`. To use Cody Gateway, your Sourcegraph instance must be connected to the service in this domain.
|
||||
|
||||
> WARNING: Sourcegraph Cody Gateway access must be included in your Sourcegraph Enterprise subscription plan. You can verify it by checking it with your account manager. If you are a [Sourcegraph Cloud](../../cloud/index.md) user, Cody is enabled by default on your instance starting with Sourcegraph 5.1.
|
||||
|
||||
## Configuring custom models
|
||||
|
||||
To configure custom models for various Cody configurations (for example, `"completions"` and `"embeddings"`), specify the desired model with the upstream provider as a prefix to the name of the model. For example, to use the `claude-2` model from Anthropic, you would configure:
|
||||
|
||||
```json
|
||||
{
|
||||
"completions": { "chatModel": "anthropic/claude-2" },
|
||||
}
|
||||
```
|
||||
|
||||
The currently supported upstream providers for models are:
|
||||
|
||||
- [`anthropic/`](https://www.anthropic.com/)
|
||||
- [`openai/`](https://openai.com/)
|
||||
|
||||
For Sourcegraph Enterprise customers, model availability depends on your Sourcegraph Enterprise subscription.
|
||||
|
||||
> WARNING: When using OpenAI models for completions, only chat completions will work - code completions are currently unsupported.
|
||||
|
||||
## Rate limits and quotas
|
||||
|
||||
Rate limits, quotas, and model availability are tied to one of the following:
|
||||
|
||||
- your Sourcegraph Enterprise product subscription for Sourcegraph Enterprise instances
|
||||
- your Sourcegraph.com account, for [Cody App users](../overview/app/index.md)
|
||||
|
||||
All successful requests to Cody Gateway will count toward your rate limits. Unsuccessful requests are not counted as usage.
|
||||
|
||||
Rate limits, quotas, and model availability are also configured per Cody feature - for example, you will have separate rate limits for Cody chat, Cody completions, and Cody embeddings.
|
||||
|
||||
In addition to the above, we may throttle concurrent requests to Cody Gateway per Sourcegraph Enterprise instance or Cody App user to prevent excessive burst consumption.
|
||||
|
||||
>NOTE: You can reach out for more details about Sourcegraph Cody Gateway access available to you and how you can gain access to higher rate limits, quotas, and/or model options.
|
||||
|
||||
## Privacy and security
|
||||
|
||||
Sourcegraph Cody Gateway does not retain sensitive data (prompt test and source code included in requests, etc.) from any traffic received. Only rate limit consumption per Sourcegraph Enterprise subscription and some high-level diagnostic data (error codes from upstream, numeric/enum request parameters, etc) are tracked.
|
||||
|
||||
The code that powers Cody Gateway is also [source-available](https://sourcegraph.com/search?q=context:global+repo:%5Egithub%5C.com/sourcegraph/sourcegraph$+f:cmd/cody-gateway+lang:go&patternType=lucky&sm=1&groupBy=path) for audit.
|
||||
|
||||
For more details about Cody Gateway security practices, please reach out to your account manager. You can also refer to the [Cody usage and privacy notice](https://about.sourcegraph.com/terms/cody-notice) for more privacy details about Cody in general.
|
||||
@ -1,80 +0,0 @@
|
||||
# Sourcegraph Cody Gateway
|
||||
|
||||
<span class="badge badge-note">Sourcegraph 5.1+</span>
|
||||
|
||||
Sourcegraph Cody Gateway powers the default `"provider": "sourcegraph"` Cody completions and embeddings for Sourcegraph Enterprise customers.
|
||||
It supports a variety of upstream LLM providers, such as [Anthropic](https://www.anthropic.com/) and [OpenAI](https://openai.com/), with rate limits, quotas, and model availability tied to your Sourcegraph Enterprise product subscription.
|
||||
[Cody App users](./../overview/app/index.md) with Sourcegraph.com accounts will also be able to use Sourcegraph Cody Gateway.
|
||||
|
||||
Reach out your account manager for more details about Sourcegraph Cody Gateway access available to you and how you can gain access to higher rate limits, quotas, and/or model options.
|
||||
|
||||
## Using Cody Gateway in Sourcegraph Enterprise
|
||||
|
||||
> WARNING: Sourcegraph Cody Gateway access must be included in your Sourcegraph Enterprise subscription plan first - reach out to your account manager for more details.
|
||||
>
|
||||
> If you are a [Sourcegraph Cloud](../../cloud/index.md) customer, Cody is enabled by default on your instance starting with Sourcegraph 5.1.
|
||||
|
||||
<span class="virtual-br"></span>
|
||||
|
||||
> NOTE: Sourcegraph Cody Gateway uses one or more third-party LLM (Large Language Model) providers. Make sure you review the [Cody usage and privacy notice](https://about.sourcegraph.com/terms/cody-notice). In particular, code snippets will be sent to a third-party language model provider when you use the Cody extension or when embeddings are enabled.
|
||||
|
||||
To enable completions and embeddings provided by Cody Gateway on your Sourcegraph Enterprise instance, simply ensure your license key is set and Cody is enabled in [site configuration](../../admin/config/site_config.md):
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"licenseKey": "<...>",
|
||||
"cody.enabled": true,
|
||||
}
|
||||
```
|
||||
|
||||
That's it! Reasonable defaults will automatically be applied, and authentication will happen automatically based on the configured license key.
|
||||
|
||||
For more details about configuring Cody, refer to the following guides:
|
||||
|
||||
- [Enabling Cody for Sourcegraph Enterprise](./../overview/enable-cody-enterprise.md)
|
||||
- [Code Graph Context: Embeddings](./code-graph.md#embeddings)
|
||||
|
||||
Cody Gateway is hosted at `cody-gateway.sourcegraph.com`. To use Cody Gateway, your Sourcegraph instance must be able to connect to the service at this domain.
|
||||
|
||||
## Configuring custom models
|
||||
|
||||
To configure custom models for various Cody configurations (e.g. `"completions"` and `"embeddings"`), specify the desired model with the upstream provider as a prefix to the name of the model. For example, to use the `claude-2` model from Anthropic, you would configure:
|
||||
|
||||
```json
|
||||
{
|
||||
"completions": { "chatModel": "anthropic/claude-2" },
|
||||
}
|
||||
```
|
||||
|
||||
The currently supported upstream providers for models are:
|
||||
|
||||
- [`anthropic/`](https://www.anthropic.com/)
|
||||
- [`openai/`](https://openai.com/)
|
||||
|
||||
For Sourcegraph Enterprise customers, model availability depends on your Sourcegraph Enterprise subscription - reach out your account manager for more details.
|
||||
|
||||
Refer to [Cody documentation](../overview/index.md) to learn more about Cody configuration.
|
||||
|
||||
> WARNING: When using OpenAI models for completions, only chat completions will work - code completions are currently unsupported.
|
||||
|
||||
## Rate limits and quotas
|
||||
|
||||
Rate limits, quotas, and model availability is tied to one of:
|
||||
|
||||
- your Sourcegraph Enterprise product subscription, for Sourcegraph Enterprise instances
|
||||
- your Sourcegraph.com account, for [Cody App users](../overview/app/index.md)
|
||||
|
||||
All successful requests to Cody Gateway will count towards your rate limits.
|
||||
Unsuccesful requests are not counted as usage.
|
||||
Rate limits, quotas, and model availability are also configured per Cody feature - for example, you will have a separate rate limits for Cody chat, Cody completions, and Cody embeddings.
|
||||
|
||||
In addition to the above, we may throttle concurrent requests to Cody Gateway per Sourcegraph Enterprise instance or Cody App user, to prevent excessive burst consumption.
|
||||
|
||||
## Privacy and security
|
||||
|
||||
Sourcegraph Cody Gateway does not retain any sensitive data (prompt test and source code included in requests, etc) from any traffic we receive.
|
||||
We only tracks rate limit consumption per Sourcegraph Enterprise subscription, and some high-level diagnostic data (errors codes from upstream, numeric/enum request parameters, etc).
|
||||
The code that powers Cody Gateway is also [source-available](https://sourcegraph.com/search?q=context:global+repo:%5Egithub%5C.com/sourcegraph/sourcegraph$+f:cmd/cody-gateway+lang:go&patternType=lucky&sm=1&groupBy=path) for audit.
|
||||
|
||||
For more details about Cody Gateway security practices, please reach out to your account manager.
|
||||
Also refer to the [Cody usage and privacy notice](https://about.sourcegraph.com/terms/cody-notice) for more privacy details about Cody in general.
|
||||
@ -50,7 +50,7 @@ Supported time units are h (hours), m ( minutes), and s (seconds).
|
||||
|
||||
## Third-party embeddings provider
|
||||
|
||||
Instead of [Sourcegraph Cody Gateway](./../cody_gateway.md), admins can also use a third-party embeddings provider like:
|
||||
Instead of [Sourcegraph Cody Gateway](./../cody-gateway.md), admins can also use a third-party embeddings provider like:
|
||||
|
||||
- OpenAI
|
||||
- Azure OpenAI <span style="margin-left:0.25rem" class="badge badge-experimental">Experimental</span>
|
||||
|
||||
@ -162,7 +162,7 @@ Files may be skipped for the following reasons:
|
||||
|
||||
### What is the default `sourcegraph` provider for completions and embeddings?
|
||||
|
||||
The default provider for completions and embeddings, specified as `"provider": "sourcegraph"` refers to the [Sourcegraph Cody Gateway](./core-concepts/cody_gateway.md). The Cody Gateway facilitates access to completions and embeddings for Sourcegraph enterprise instances by leveraging third-party services such as Anthropic and OpenAI.
|
||||
The default provider for completions and embeddings, specified as `"provider": "sourcegraph"` refers to the [Sourcegraph Cody Gateway](./core-concepts/cody-gateway.md). The Cody Gateway facilitates access to completions and embeddings for Sourcegraph enterprise instances by leveraging third-party services such as Anthropic and OpenAI.
|
||||
|
||||
### What third-party cloud services does Cody depend on?
|
||||
|
||||
@ -170,7 +170,7 @@ Cody relies on one primary third-party dependency, i.e., Anthropic's Claude API.
|
||||
|
||||
Additionally, Cody can optionally use OpenAI for generating embeddings, enhancing the quality of its context snippets, although this is not mandatory.
|
||||
|
||||
It's worth noting that these dependencies remain consistent when utilizing the [default `sourcegraph` provider, Cody Gateway](./core-concepts/cody_gateway.md), which uses the same third-party providers.
|
||||
It's worth noting that these dependencies remain consistent when utilizing the [default `sourcegraph` provider, Cody Gateway](./core-concepts/cody-gateway.md), which uses the same third-party providers.
|
||||
|
||||
### What is the retention policy for Anthropic and OpenAI?
|
||||
|
||||
|
||||
@ -139,7 +139,7 @@ After a successful installation, follow these steps to complete the app setup:
|
||||
|
||||
## Rate limiting
|
||||
|
||||
Several forms of rate limiting help us manage costs for free versions of Cody. We're working actively to relax these limits and facilitate users with [Cody Gateway](./../../core-concepts/cody_gateway.md).
|
||||
Several forms of rate limiting help us manage costs for free versions of Cody. We're working actively to relax these limits and facilitate users with [Cody Gateway](./../../core-concepts/cody-gateway.md).
|
||||
|
||||
If you hit these limits, you can request an increased limit by visiting our [Discord](https://discord.com/servers/sourcegraph-969688426372825169) channel for both chats and completions. If you'd like to use your third-party LLM provider instead of Cody Gateway, create your key with Anthropic or OpenAI and [update your app configuration](app_configuration.md).
|
||||
|
||||
|
||||
@ -119,7 +119,7 @@ Cody enhances your coding experience by providing intelligent code suggestions,
|
||||
### Prerequisites
|
||||
|
||||
- You have Sourcegraph version 5.1.0 or above
|
||||
- A Sourcegraph enterprise subscription with [Cody Gateway access](./../core-concepts/cody_gateway.md) or [an account with a third-party LLM provider](#using-a-third-party-llm-provider-directly)
|
||||
- A Sourcegraph enterprise subscription with [Cody Gateway access](./../core-concepts/cody-gateway.md) or [an account with a third-party LLM provider](#using-a-third-party-llm-provider-directly)
|
||||
|
||||
### Enable Cody on your Sourcegraph instance
|
||||
|
||||
@ -127,7 +127,7 @@ Cody uses one or more third-party LLM (Large Language Model) providers. Make sur
|
||||
|
||||
This requires site-admin privileges. To do so,
|
||||
|
||||
1. First, configure your desired LLM provider either by [Using Sourcegraph Cody Gateway](./../core-concepts/cody_gateway.md#using-cody-gateway-in-sourcegraph-enterprise) (recommended) or [Using a third-party LLM provider directly](#using-a-third-party-llm-provider-directly)
|
||||
1. First, configure your desired LLM provider either by [Using Sourcegraph Cody Gateway](./../core-concepts/cody-gateway.md#using-cody-gateway-in-sourcegraph-enterprise) (recommended) or [Using a third-party LLM provider directly](#using-a-third-party-llm-provider-directly)
|
||||
|
||||
> NOTE: If you are a Sourcegraph Cloud customer, skip directly to step 3.
|
||||
|
||||
@ -224,7 +224,7 @@ To enable Cody only for some users, for example, when rolling out a Cody POC, fo
|
||||
|
||||
## Using a third-party LLM provider
|
||||
|
||||
Instead of [Sourcegraph Cody Gateway](./../core-concepts/cody_gateway.md), you can also configure Sourcegraph to use a third-party provider directly, like:
|
||||
Instead of [Sourcegraph Cody Gateway](./../core-concepts/cody-gateway.md), you can also configure Sourcegraph to use a third-party provider directly, like:
|
||||
|
||||
- Anthropic
|
||||
- OpenAI
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
# How to set up Cody Gateway locally
|
||||
|
||||
> WARNING: This is a development guide - to use Cody Gateway for Sourcegraph, refer to [Sourcegraph Cody Gateway](../../cody/core-concepts/cody_gateway.md).
|
||||
> WARNING: This is a development guide - to use Cody Gateway for Sourcegraph, refer to [Sourcegraph Cody Gateway](../../cody/core-concepts/cody-gateway.md).
|
||||
|
||||
This guide documents how to set up [Cody Gateway](https://handbook.sourcegraph.com/departments/engineering/teams/cody/cody-gateway/) locally for development.
|
||||
|
||||
|
||||
@ -44,7 +44,7 @@
|
||||
<!-- [Test a Phabricator and Gitolite instance](test_phabricator.md) -->
|
||||
|
||||
|
||||
## Maintenance
|
||||
## Maintenance
|
||||
|
||||
- [How to update pnpm to a newer version](update_pnpm.md)
|
||||
|
||||
@ -79,5 +79,4 @@
|
||||
|
||||
Running Sourcegraph on Windows is not actively tested, but should be possible within the Windows Subsystem for Linux (WSL).
|
||||
Sourcegraph currently relies on Unix specifics in several places, which makes it currently not possible to run Sourcegraph directly inside Windows without WSL.
|
||||
We are happy to accept contributions here! :)
|
||||
|
||||
We are happy to accept contributions here!
|
||||
|
||||
Loading…
Reference in New Issue
Block a user