Commit Graph

25 Commits

Author SHA1 Message Date
Chris Wendt
044ac2edd5
docs: Set dir in mermaid scripts (#29082) 2021-12-15 18:58:14 -07:00
Warren Gifford
8f169b4b1c
corrected deadlinks for code appendix in hover docs (#27402)
* corrected deadlinks for code appendix in hover docs

* changed links back to a search rather than direct links to code lines
2021-11-11 14:04:03 -08:00
Ole Kröger
569c822c49
changed from localhost to docs.sourcegraph.com (#19339) 2021-10-07 17:01:45 +02:00
Stephen Gutekanst
9a5e7e2c34
API docs: codeintel: begin indexing API docs for search (#25720)
This PR updates the codeintel worker, and lsifstore, to begin actually writing API docs data to the new `lsif_data_documentation_search_public` and `lsif_data_documentation_search_private` Postgres FTS tables when new LSIF bundles are uploaded.

This is on by default, but can be disabled via the new site config feature flag `"apidocs.search.indexing": "disabled"`.

These tables are for _API docs search indexing only_, and to prevent any scaling issues / not break the DB, the default configuration `"apidocs.search.index-size-limit-factor": 1.0` limits the size of each table independently to 250 million symbols (rows) (500 million across both tables, approx. 12.5k Go repos total). This was arrived at through [some estimation documented in the architecture design doc.](https://github.com/sourcegraph/sourcegraph/blob/main/doc/dev/background-information/codeintel/apidocs/index.md#scaling-estimation)

For more details see [the architecture docs](https://github.com/sourcegraph/sourcegraph/blob/main/doc/dev/background-information/codeintel/apidocs/index.md#search-architecture) and [the PR description](https://github.com/sourcegraph/sourcegraph/pull/25720#issue-750526032).

Signed-off-by: Stephen Gutekanst <stephen@sourcegraph.com>
2021-10-06 11:56:22 -07:00
Stephen Gutekanst
fdff738030
backout API docs changes (#25715)
* Revert "API docs: codeintel: add OOB migration to index API docs for search (#25207)"

This reverts commit b910fedad1.

* Revert "API docs: codeintel: begin indexing API docs for search (#25666)"

This reverts commit 30f0c79420.
2021-10-05 21:07:47 +00:00
Stephen Gutekanst
30f0c79420
API docs: codeintel: begin indexing API docs for search (#25666)
This brings back #25206 - which was reverted in #25315 due to an issue with querying from the wrong DB (missed in dev, since codeintel and main app DBs are the same in dev this was an easy mistake to make despite me consciously trying to avoid it.)

First commit brings back the change in original state; next commits fix the actual underlying issue:

Instead of relying on the lsif_dumps table, which is in the main app DB, to locate the dumps we should delete (i.e. all rows which came from a dump for the same upload root and LSIF indexer/language), we instead introduce a dump_root column which duplicates this info on the lsif_data_documentation_search_* tables.

This is duplicative/repetitive data across rows, similar to other rows in this table. This is not optimal, but the size of the table is constrained, the duplicated data is very small, and I will improve this later once queries against these tables have solidified a bit.

Signed-off-by: Stephen Gutekanst <stephen@sourcegraph.com>
2021-10-05 11:58:11 -07:00
Eric Fritz
95cbf3c85a
codeintel-qa: Rewrite precise-code-intel-tester (#25412) 2021-10-04 09:46:24 -05:00
Erik Seliger
fd9b5eebdf
Revert "API docs: codeintel: begin indexing API docs for search (#252… (#25315)
This reverts commit d330de0f61.
2021-09-23 14:45:26 -07:00
Stephen Gutekanst
d330de0f61
API docs: codeintel: begin indexing API docs for search (#25206)
* API docs: codeintel: begin indexing API docs for search

Depends on #25198 and #25199

Begins writing API docs data to the new `lsif_data_documentation_search_*` tables when
new LSIF bundles are uploaded. This is on by default, but can be disabled via the new
site config feature flag `"apidocs.search.indexing": "disabled"`.

See the architecture docs added in #25197 for more details, and #25199 for details on
the DB schema itself.

Signed-off-by: Stephen Gutekanst <stephen@sourcegraph.com>
2021-09-23 00:02:15 +00:00
Stephen Gutekanst
62378067ca
doc/dev: document API docs search indexing architecture, tradeoffs, scaling, etc. (#25197)
Signed-off-by: Stephen Gutekanst <stephen@sourcegraph.com>
2021-09-21 16:49:33 -07:00
Stephen Gutekanst
35a0c87b43
doc/dev: add API docs architecture docs (#25195)
Signed-off-by: Stephen Gutekanst <stephen@sourcegraph.com>
2021-09-21 23:17:28 +00:00
Eric Fritz
c4e0685e3e
doc: Misc updates to codeintel development docs (#23916) 2021-08-13 13:15:10 -05:00
Stefan Hengl
c61b0f1a7b
all: /bin/bash -> /usr/bin/env bash (#23673)
I ran into issues setting up Sourcegraph on NixOS, because NixOS, like
some other distros, doesn't have a /bin/bash. We already use /usr/bin/env
in many of our scripts, so this improves consistency, too.
2021-08-06 12:02:43 +02:00
TJ DeVries
0a5347213c
feat(code-intel): Auto index repogroups (#21982)
* START_OF_PR: Move autoindexing to repogropus

* migration: add down migration

* remove RepoUsageStatistics from everywhere I can find

* add autoindex enable and disable support part 1

* push for eric

* fixup: change repogroups return type

* fixup: fix most of eric's comments

* fixup: rebase and get new generate stuff

* hmmm somehow i typed generake in a diff?

* start removing more references to indexable_repos wherever I can

* oops, missing one

* more fixups and tests

* last fixups, next buildkite

* missed some generates

* dev lint is cool

* restore discovery file cause i broke tests

* fixup: add back function I didnt mean to delete

* fixup from last review

* final fixup maybe

* generate
2021-06-17 11:35:15 -04:00
Eric Fritz
b574a2878d
doc: Update code-intel sequence diagrams (#21321) 2021-05-25 10:17:29 -05:00
Stephen Gutekanst
e3d0a85eac
doc/dev: link to precise-code-intel-tester 2021-05-10 17:59:54 -07:00
Eric Fritz
b3c85cd4d9
chore: Update codenotify for code intelligence docs (#18455) 2021-02-18 21:02:33 -06:00
Nick Snyder
691671da2d
High level architecture docs (#16886) 2021-01-20 19:49:54 +01:00
Ólafur Páll Geirsson
46953498dc
Fix links to 404 page (#17366)
Previously, Sourcegraph linked to a non-existent page on "basic code
intelligence" when hovering over a tooltip about code intel. Now, the
link points to the page on "search-based code intelligence".
2021-01-19 11:44:35 +01:00
Eric Fritz
fbd965f68a
doc: Update codeintel indexer docs (#15818) 2020-11-18 12:25:54 -06:00
Eric Fritz
6dc85f375b
doc: Update codeintel sequence diagrams (#15200) 2020-10-30 17:17:53 -05:00
Christina Forney
e4958b671b
Elevate product areas in documentation (#14792) 2020-10-18 10:55:18 -07:00
Christina Forney
96264ee1fb
Organizing code intel docs into explanations, tutorials, how-tos, and references (#14642) 2020-10-13 21:41:09 -07:00
Eric Fritz
d807558c4e
doc/dev: Update architecture diagram (#14633) 2020-10-13 11:05:44 -05:00
Christina Forney
5ec6c99a99
Documentation reorg: Development docs (#14509) 2020-10-07 22:13:59 -07:00