sourcegraph/dev/managedservicesplatform
Robert Lin 97ab982d0e
dev/msp: add BigQuery support (#58541)
This change adds support for provisioning a BigQuery dataset and tables for a service to write to, similar to how we set up Cody Gateway. For now, we provision only a single BigQuery table - the intent is to allow services to generate their own telemetry events, which should only have a single table.

For schema configuration, we ask that it be provided as a JSON file alongside the spec, i.e.

```
service.yaml
${mytable}.bigquerytable.json
```

Basic configuration just looks like this:

```yaml
environments:
- id: ...
  # ...
  resources:
    bigQueryDataset:
      tables:
        - "example"
```

Example diff: https://github.com/sourcegraph/managed-services/pull/116

Usage:

```go
		bq, err := contract.BigQuery.GetTableWriter(ctx, "example")
		if err != nil {
			return nil, errors.Wrap(err, "BigQuery.GetTableWriter")
		}
		if err := bq.Write(ctx, BigQueryEntry{
			Name:      "service.started",
			CreatedAt: time.Now(),
		}); err != nil {
			return nil, errors.Wrap(err, "bq.Write")
		}
```

For usage, we wrap `bigquery.Inserter` using a helper that guides users to implement `bigquery.ValueSaver`, which is what we do in Cody Gateway.

This change also removes the service-env-var prefix, which I've decided might be rather useless :) it's not currently used at all.

Closes https://github.com/sourcegraph/sourcegraph/issues/56849
2023-12-04 18:40:18 -08:00
..
googlesecretsmanager dev/sg/msp: fixes for end-to-end prototype (#56608) 2023-09-14 18:15:04 +00:00
internal dev/msp: add BigQuery support (#58541) 2023-12-04 18:40:18 -08:00
spec dev/msp: add BigQuery support (#58541) 2023-12-04 18:40:18 -08:00
terraformcloud msp: add artificial operators team to generated workspaces (#58509) 2023-11-23 11:04:54 -08:00
BUILD.bazel dev/msp: migrate service accounts, IAM to 'iam' module (#58216) 2023-11-08 18:03:03 -08:00
cdktf.go dev/msp: extract resolved image tag to tfvars, add stable generate mode (#58180) 2023-11-08 09:40:39 -08:00
CODENOTIFY chore: update CODENOTIFY for Joe (#56647) 2023-09-14 16:09:33 +00:00
managedservicesplatform.go dev/msp: migrate service accounts, IAM to 'iam' module (#58216) 2023-11-08 18:03:03 -08:00