mirror of
https://github.com/sourcegraph/sourcegraph.git
synced 2026-02-06 17:31:43 +00:00
b8e31fde27
When a text input is submitted for generating embeddings the response may be null. If we attempt retries and still cannot generate embeddings for this input text then we return an error which calls for failing the entire embed repo job. [Slack thread](https://sourcegraph.slack.com/archives/C053L1AQ0BC/p1688676751106069) [Issue](https://github.com/sourcegraph/sourcegraph/issues/55469) This PR introduces a configuration `ExcludeChunkOnError`. When set to true an embed repo job will proceed with the rest of the embed repo job when these generate embeddings errors occur. However, the file that generated the input text which received an error is excluded from the index as to avoid partially indexing the file. I'll add more details on the first iteration of this solution and the trade offs in a separate comment. ## Test plan <!-- All pull requests REQUIRE a test plan: https://docs.sourcegraph.com/dev/background-information/testing_principles --> Embed test cases added |
||
|---|---|---|
| .. | ||
| aws_codecommit.schema.json | ||
| azuredevops.schema.json | ||
| batch_spec.schema.json | ||
| bitbucket_cloud.schema.json | ||
| bitbucket_server_util.go | ||
| bitbucket_server.schema.json | ||
| bitbucketcloud_util.go | ||
| BUILD.bazel | ||
| changeset_spec.schema.json | ||
| extension_schema.go | ||
| gen.go | ||
| gen.sh | ||
| gerrit.schema.json | ||
| github_util.go | ||
| github.schema.json | ||
| gitlab_util.go | ||
| gitlab.schema.json | ||
| gitolite.schema.json | ||
| go-modules.schema.json | ||
| json-schema-draft-07.schema.json | ||
| jvm-packages.schema.json | ||
| localgit.schema.json | ||
| npm-packages.schema.json | ||
| other_external_service.schema.json | ||
| package.json | ||
| pagure.schema.json | ||
| perforce.schema.json | ||
| phabricator.schema.json | ||
| python-packages.schema.json | ||
| README.md | ||
| ruby-packages.schema.json | ||
| rust-packages.schema.json | ||
| schema.go | ||
| settings.schema.json | ||
| site.schema.json | ||
| stringdata.go | ||
| tsconfig.json | ||
| validation_test.go | ||
Sourcegraph JSON Schemas
JSON Schema is a way to define the structure of a JSON document. It enables typechecking and code intelligence on JSON documents.
Sourcegraph uses the following JSON Schemas:
Modifying a schema
- Edit the
*.schema.jsonfile in this directory. - Run
go generateto update the*_stringdata.jsonfile. - Commit the changes to both files.
- Run
sg startto automatically update TypeScript schema files.
Known issues
- The JSON Schema IDs (URIs) are of the form
https://sourcegraph.com/v1/*.schema.json#, but these are not actually valid URLs. This means you generally need to supply them to JSON Schema validation libraries manually instead of having the validator fetch the schema from the web.