mirror of
https://github.com/sourcegraph/sourcegraph.git
synced 2026-02-06 16:51:55 +00:00
Code AI platform with Code Search & Cody
e835a66c76
This change adds: - telemetry export background jobs: flagged behind `TELEMETRY_GATEWAY_EXPORTER_EXPORT_ADDR`, default empty => disabled - telemetry redaction: configured in package `internal/telemetry/sensitivemetadataallowlist` - telemetry-gateway service receiving events and forwarding it to a pub/sub topic (or just logging it, as configured in local dev) - utilities for easily creating an event recorder: `internal/telemetry/telemetryrecorder` Notes: - all changes are feature-flagged to some degree, off by default, so the merge should be fairly low-risk. - we decided that transmitting the full license key continues to be the way to go. we transmit it once per stream and attach it on all events in the telemetry-gateway. there is no auth mechanism at the moment - GraphQL return type `EventLog.Source` is now a plain string instead of string enum. This should not be a breaking change in our clients, but must be made so that our generated V2 events do not break requesting of event logs Stacked on https://github.com/sourcegraph/sourcegraph/pull/56520 Closes https://github.com/sourcegraph/sourcegraph/issues/56289 Closes https://github.com/sourcegraph/sourcegraph/issues/56287 ## Test plan Add an override to make the export super frequent: ``` env: TELEMETRY_GATEWAY_EXPORTER_EXPORT_INTERVAL: "10s" TELEMETRY_GATEWAY_EXPORTER_EXPORTED_EVENTS_RETENTION: "5m" ``` Start sourcegraph: ``` sg start ``` Enable `telemetry-export` featureflag (from https://github.com/sourcegraph/sourcegraph/pull/56520) Emit some events in GraphQL: ```gql mutation { telemetry { recordEvents(events:[{ feature:"foobar" action:"view" source:{ client:"WEB" } parameters:{ version:0 } }]) { alwaysNil } } ``` See series of log events: ``` [ worker] INFO worker.telemetrygateway-exporter telemetrygatewayexporter/telemetrygatewayexporter.go:61 Telemetry Gateway export enabled - initializing background routines [ worker] INFO worker.telemetrygateway-exporter telemetrygatewayexporter/exporter.go:99 exporting events {"maxBatchSize": 10000, "count": 1} [telemetry-g...y] INFO telemetry-gateway.pubsub pubsub/topic.go:115 Publish {"TraceId": "7852903434f0d2f647d397ee83b4009b", "SpanId": "8d945234bccf319b", "message": "{\"event\":{\"id\":\"dc96ae84-4ac4-4760-968f-0a0307b8bb3d\",\"timestamp\":\"2023-09-19T01:57:13.590266Z\",\"feature\":\"foobar\", .... ``` Build: ``` export VERSION="insiders" bazel run //cmd/telemetry-gateway:candidate_push --config darwin-docker --stamp --workspace_status_command=./dev/bazel_stamp_vars.sh -- --tag $VERSION --repository us.gcr.io/sourcegraph-dev/telemetry-gateway ``` Deploy: https://github.com/sourcegraph/managed-services/pull/7 Add override: ```yaml env: # Port required. TODO: What's the best way to provide gRPC addresses, such that a # localhost address is also possible? TELEMETRY_GATEWAY_EXPORTER_EXPORT_ADDR: "https://telemetry-gateway.sgdev.org:443" ``` Repeat the above (`sg start` and emit some events): ``` [ worker] INFO worker.telemetrygateway-exporter telemetrygatewayexporter/exporter.go:94 exporting events {"maxBatchSize": 10000, "count": 6} [ worker] INFO worker.telemetrygateway-exporter telemetrygatewayexporter/exporter.go:113 events exported {"maxBatchSize": 10000, "succeeded": 6} [ worker] INFO worker.telemetrygateway-exporter telemetrygatewayexporter/exporter.go:94 exporting events {"maxBatchSize": 10000, "count": 1} [ worker] INFO worker.telemetrygateway-exporter telemetrygatewayexporter/exporter.go:113 events exported {"maxBatchSize": 10000, "succeeded": 1} ``` |
||
|---|---|---|
| .aspect | ||
| .buildkite | ||
| .github | ||
| .vscode | ||
| client | ||
| cmd | ||
| dev | ||
| doc | ||
| docker-images | ||
| enterprise | ||
| internal | ||
| lib | ||
| migrations | ||
| monitoring | ||
| schema | ||
| src-tauri | ||
| testing | ||
| third_party | ||
| third-party-licenses | ||
| ui/assets | ||
| windows/installer/cody | ||
| wolfi-images | ||
| wolfi-packages | ||
| .bazel_fix_commands.json | ||
| .bazelignore | ||
| .bazeliskrc | ||
| .bazelrc | ||
| .bazelversion | ||
| .browserslistrc | ||
| .dockerignore | ||
| .editorconfig | ||
| .eslintignore | ||
| .eslintrc.js | ||
| .gitattributes | ||
| .gitignore | ||
| .graphqlrc.yml | ||
| .hadolint.yaml | ||
| .mailmap | ||
| .mocharc.js | ||
| .npmrc | ||
| .percy.yml | ||
| .pre-commit-config.yaml | ||
| .prettierignore | ||
| .stylelintignore | ||
| .stylelintrc.json | ||
| .tool-versions | ||
| .trivyignore | ||
| babel.config.jest.js | ||
| babel.config.js | ||
| BUILD.bazel | ||
| CHANGELOG.app.md | ||
| CHANGELOG.md | ||
| CODENOTIFY | ||
| CONTRIBUTING.md | ||
| deps.bzl | ||
| doc.go | ||
| eslint-relative-formatter.js | ||
| flake.lock | ||
| flake.nix | ||
| gen.go | ||
| go.mod | ||
| go.sum | ||
| graphql-schema-linter.config.js | ||
| gulpfile.js | ||
| jest.config.base.js | ||
| jest.config.js | ||
| jest.snapshot-resolver.js | ||
| junit.xml | ||
| LICENSE | ||
| LICENSE.enterprise | ||
| lighthouserc.js | ||
| linter_deps.bzl | ||
| mockgen.temp.yaml | ||
| mockgen.test.yaml | ||
| mockgen.yaml | ||
| nogo_config.json | ||
| package.json | ||
| pnpm-lock.yaml | ||
| pnpm-workspace.yaml | ||
| postcss.config.js | ||
| prettier.config.js | ||
| README.md | ||
| renovate.json | ||
| SECURITY.md | ||
| service-catalog.yaml | ||
| sg.config.yaml | ||
| shell.nix | ||
| sonar-project.properties | ||
| stamp_tags.bzl | ||
| svgo.config.js | ||
| tsconfig.all.json | ||
| tsconfig.base.json | ||
| tsconfig.eslint.json | ||
| WORKSPACE | ||
Docs •
Contributing •
Twitter •
Discord
Sourcegraph makes it easy to read, write, and fix code—even in big, complex codebases.
- Code search: Search all of your repositories across all branches and all code hosts.
- Code intelligence: Navigate code, find references, see code owners, trace history, and more.
- Fix and refactor: Roll out large-scale changes to many repositories at once and track big migrations.
Getting started
- Download Sourcegraph for macOS and Linux
- Use Sourcegraph on the cloud or self-hosted
- Sourcegraph.com public code search
Development
Refer to the Developing Sourcegraph guide to get started.
Documentation
The doc directory has additional documentation for developing and understanding Sourcegraph:
- Project FAQ
- Architecture: high-level architecture
- Database setup: database best practices
- Go style guide
- Documentation style guide
- GraphQL API: useful tips when modifying the GraphQL API
- Contributing
License
This repository contains primarily non-OSS-licensed files, and some OSS licensed files (for example, IDE extensions). We maintain one repository rather than two separate repositories mainly for development convenience.
See the LICENSE for an explanation of how files are licensed.