mirror of
https://github.com/sourcegraph/sourcegraph.git
synced 2026-02-06 20:11:54 +00:00
e835a66c76
This change adds: - telemetry export background jobs: flagged behind `TELEMETRY_GATEWAY_EXPORTER_EXPORT_ADDR`, default empty => disabled - telemetry redaction: configured in package `internal/telemetry/sensitivemetadataallowlist` - telemetry-gateway service receiving events and forwarding it to a pub/sub topic (or just logging it, as configured in local dev) - utilities for easily creating an event recorder: `internal/telemetry/telemetryrecorder` Notes: - all changes are feature-flagged to some degree, off by default, so the merge should be fairly low-risk. - we decided that transmitting the full license key continues to be the way to go. we transmit it once per stream and attach it on all events in the telemetry-gateway. there is no auth mechanism at the moment - GraphQL return type `EventLog.Source` is now a plain string instead of string enum. This should not be a breaking change in our clients, but must be made so that our generated V2 events do not break requesting of event logs Stacked on https://github.com/sourcegraph/sourcegraph/pull/56520 Closes https://github.com/sourcegraph/sourcegraph/issues/56289 Closes https://github.com/sourcegraph/sourcegraph/issues/56287 ## Test plan Add an override to make the export super frequent: ``` env: TELEMETRY_GATEWAY_EXPORTER_EXPORT_INTERVAL: "10s" TELEMETRY_GATEWAY_EXPORTER_EXPORTED_EVENTS_RETENTION: "5m" ``` Start sourcegraph: ``` sg start ``` Enable `telemetry-export` featureflag (from https://github.com/sourcegraph/sourcegraph/pull/56520) Emit some events in GraphQL: ```gql mutation { telemetry { recordEvents(events:[{ feature:"foobar" action:"view" source:{ client:"WEB" } parameters:{ version:0 } }]) { alwaysNil } } ``` See series of log events: ``` [ worker] INFO worker.telemetrygateway-exporter telemetrygatewayexporter/telemetrygatewayexporter.go:61 Telemetry Gateway export enabled - initializing background routines [ worker] INFO worker.telemetrygateway-exporter telemetrygatewayexporter/exporter.go:99 exporting events {"maxBatchSize": 10000, "count": 1} [telemetry-g...y] INFO telemetry-gateway.pubsub pubsub/topic.go:115 Publish {"TraceId": "7852903434f0d2f647d397ee83b4009b", "SpanId": "8d945234bccf319b", "message": "{\"event\":{\"id\":\"dc96ae84-4ac4-4760-968f-0a0307b8bb3d\",\"timestamp\":\"2023-09-19T01:57:13.590266Z\",\"feature\":\"foobar\", .... ``` Build: ``` export VERSION="insiders" bazel run //cmd/telemetry-gateway:candidate_push --config darwin-docker --stamp --workspace_status_command=./dev/bazel_stamp_vars.sh -- --tag $VERSION --repository us.gcr.io/sourcegraph-dev/telemetry-gateway ``` Deploy: https://github.com/sourcegraph/managed-services/pull/7 Add override: ```yaml env: # Port required. TODO: What's the best way to provide gRPC addresses, such that a # localhost address is also possible? TELEMETRY_GATEWAY_EXPORTER_EXPORT_ADDR: "https://telemetry-gateway.sgdev.org:443" ``` Repeat the above (`sg start` and emit some events): ``` [ worker] INFO worker.telemetrygateway-exporter telemetrygatewayexporter/exporter.go:94 exporting events {"maxBatchSize": 10000, "count": 6} [ worker] INFO worker.telemetrygateway-exporter telemetrygatewayexporter/exporter.go:113 events exported {"maxBatchSize": 10000, "succeeded": 6} [ worker] INFO worker.telemetrygateway-exporter telemetrygatewayexporter/exporter.go:94 exporting events {"maxBatchSize": 10000, "count": 1} [ worker] INFO worker.telemetrygateway-exporter telemetrygatewayexporter/exporter.go:113 events exported {"maxBatchSize": 10000, "succeeded": 1} ``` |
||
|---|---|---|
| .. | ||
| api | ||
| batches | ||
| cliutil | ||
| codeintel | ||
| errors | ||
| gitservice | ||
| iterator | ||
| output | ||
| pointers | ||
| process | ||
| servicecatalog | ||
| go.mod | ||
| go.sum | ||
| README.md | ||
| tools.go | ||
Sourcegraph lib module
This directory is the root of a separate go module from the primary module rooted at sourcegraph/sourcegraph. This module exists to hold code that we want to reuse outside of the sourcegraph/sourcegraph repo.
Code in this module should not import from sourcegraph/sourcegraph or from other Sourcegraph repositories to avoid complicated dependency relationships. Instead consider moving code from elsewhere into this module.