sourcegraph/doc/dev/background-information/codeintel
Stephen Gutekanst 9a5e7e2c34
API docs: codeintel: begin indexing API docs for search (#25720)
This PR updates the codeintel worker, and lsifstore, to begin actually writing API docs data to the new `lsif_data_documentation_search_public` and `lsif_data_documentation_search_private` Postgres FTS tables when new LSIF bundles are uploaded.

This is on by default, but can be disabled via the new site config feature flag `"apidocs.search.indexing": "disabled"`.

These tables are for _API docs search indexing only_, and to prevent any scaling issues / not break the DB, the default configuration `"apidocs.search.index-size-limit-factor": 1.0` limits the size of each table independently to 250 million symbols (rows) (500 million across both tables, approx. 12.5k Go repos total). This was arrived at through [some estimation documented in the architecture design doc.](https://github.com/sourcegraph/sourcegraph/blob/main/doc/dev/background-information/codeintel/apidocs/index.md#scaling-estimation)

For more details see [the architecture docs](https://github.com/sourcegraph/sourcegraph/blob/main/doc/dev/background-information/codeintel/apidocs/index.md#search-architecture) and [the PR description](https://github.com/sourcegraph/sourcegraph/pull/25720#issue-750526032).

Signed-off-by: Stephen Gutekanst <stephen@sourcegraph.com>
2021-10-06 11:56:22 -07:00
..
apidocs API docs: codeintel: begin indexing API docs for search (#25720) 2021-10-06 11:56:22 -07:00
diagrams doc: Misc updates to codeintel development docs (#23916) 2021-08-13 13:15:10 -05:00
auto-indexing.md doc: Misc updates to codeintel development docs (#23916) 2021-08-13 13:15:10 -05:00
CODENOTIFY chore: Update codenotify for code intelligence docs (#18455) 2021-02-18 21:02:33 -06:00
deployment.md doc: Misc updates to codeintel development docs (#23916) 2021-08-13 13:15:10 -05:00
extensions.md doc: Update code-intel sequence diagrams (#21321) 2021-05-25 10:17:29 -05:00
index.md codeintel-qa: Rewrite precise-code-intel-tester (#25412) 2021-10-04 09:46:24 -05:00
queries.md doc: Misc updates to codeintel development docs (#23916) 2021-08-13 13:15:10 -05:00
uploads.md doc: Update code-intel sequence diagrams (#21321) 2021-05-25 10:17:29 -05:00