| .. | ||
| docs | ||
| src | ||
| test-data | ||
| .eslintrc.js | ||
| babel.config.js | ||
| build.sh | ||
| Dockerfile | ||
| jest.config.js | ||
| jest.setup.js | ||
| main.go | ||
| package.json | ||
| prometheus.yml | ||
| README.md | ||
| tsconfig.json | ||
| tslint.json | ||
| yarn.lock | ||
Precise code intelligence system
This project is an early adopter of Microsoft's LSIF standard. LSIF (Language Server Index Format) is a format used to store the results of language server queries that are computed ahead-of-time. We uses this format to provide jump-to-definition, find-reference, and hover docstring functionality.
LSIF code intelligence
LSIF dumps are generated by running an LSIF indexer in a build or continuous integration environment. The dump is uploaded to a Sourcegraph instance via Sourcegraph CLI. An LSIF API server, proxied by the frontend for auth, answers relevant LSP queries to provide fast and precise code intelligence.
Architecture
This project is split into three parts, all currently written in TypeScript. These parts are deployable independently but can also run in the same docker container to simplify deployment with docker-compose.
- The API server receives LSIF uploads and answers LSP queries via HTTP.
- The bundle-manager answers queries about a particular upload by looking at relevant SQLite databases on-disk.
- The worker dequeues unconverted LSIF uploads from Postgres and converts them into SQLite databases that can be queried by the server.
Documentation
- Usage documentation is provided on Sourcegraph.com.
- API endpoint documentation is provided in api.md.
- Database configuration and migrations are described in database.md.
- Data models are described in datamodel.md and datamodel.pg.md.
Entrypoint
The docker image for this part of the application wraps a server, a bundle manager, and a worker in a goreman supervisor. By default, there will be one API process and one worker process. The number of replicas per process can be tuned with the environment variables PRECISE_CODE_INTEL_NUM_APIS (zero or more), PRECISE_CODE_INTEL_NUM_BUNDLE_MANAGERS (zero or one), and PRECISE_CODE_INTEL_NUM_WORKERS (zero or more).
Prometheus metrics
The precise-code-intel-api-server and precise-code-intel-bundle-manager expose HTTP APIs on ports 3186 and 3187, respectively. These APIs contain a /metrics endpoint to be scraped by Prometheus. The precise-code-intel-worker exposes a metrics server (but nothing else interesting) on port 3188. It's possible to run multiple workers, but impossible for them all to serve metrics from the same port. Therefore, this container also includes a minimally-configured Prometheus process that will scrape metrics from all of the processes. It is suggested that you use federation to scrape all of the process metrics at once instead of scraping the individual ports directly. Doing so will ensure that scaling up or down the number of workers will not change the the required Prometheus configuration.