sourcegraph/lib/go.mod

147 lines
6.5 KiB
Modula-2
Raw Permalink Normal View History

module github.com/sourcegraph/sourcegraph/lib
go 1.22
toolchain go1.22.4
require (
lib/enterpriseportal: initial service API for RFC 885 (#62263) See [RFC 885 Sourcegraph Enterprise Portal (go/enterprise-portal)](https://docs.google.com/document/d/1tiaW1IVKm_YSSYhH-z7Q8sv4HSO_YJ_Uu6eYDjX7uU4/edit#heading=h.tdaxc5h34u7q) - closes CORE-6. The only files requiring in-depth review are the `.proto` files, as everything else is generated: - `lib/enterpriseportal/subscriptions/v1/subscriptions.proto` - `lib/enterpriseportal/codyaccess/v1/codyaccess.proto` This PR only introduces API definitions - implementation will come as subsequent PRs, tracked in the ["Launch Enterprise Portal" Linear project](https://linear.app/sourcegraph/project/launch-sourcegraph-enterprise-portal-ee5d9ea105c2). Before reviewing the diffs, **please review this PR description in depth**. ### Design goals This initial schema aims to help achieve CORE-97 by adding our initial "get subscription Cody access", as well our general Stage 1 goal of providing read-only access to our existing enterprise subscription mechanisms. In doing so, we can start to reshape the API in a way that accommodates future growth and addresses some debt we have accumulated over time, before the Stage 2 goal of having the new Enterprise Portal be the source-of-truth for all things subscriptions. I am also aiming for a conservative approach with the Cody Gateway access related RPCs, to ease migration risks and allow for Cody teams to follow up quickly with more drastic changes in a V2 of the service after a Core-Services-driven migration to use the new service: https://github.com/sourcegraph/sourcegraph/pull/62263#issuecomment-2101874114 ### Design overview - **Multiple services**: Enterprise Portal aims to be the home of most Enterprise-related subscription and access management, but each component should be defined as a separate service to maintain clear boundaries between "core" capabilities and future extensions. One problem we see in the `dotcom { productSubscriptions }` is the embedding of additional concepts like Cody Gateway access makes the API surface unwieldy and brittle, and encourages an internal design that bundles everything together (the `product_subscriptions` table has 10 `cody_gateway_*` columns today). More concretely, this PR designs 2 services that Enterprise Portal will implement: - `EnterprisePortalSubscriptionsService` (`subscriptions.proto`): subscriptions and licenses CRUD - `EnterprisePortalCodyGatewayService` (`codygateway.proto`): Enterprise Cody Gateway access - **Multiple protocols**: We use [ConnectRPC](https://connectrpc.com/) to generate traditional gRPC handlers for service-to-service use, but also a plain HTTP/1 "REST"-ish protocol (the ["Connect Protocol"](https://connectrpc.com/docs/protocol)) that works for web clients and simple integrations. Go bindings for the Connect protocol are generated into the `v1connect` subpackages. - **Future licensing model/mechanism changes**: The _Subscription_ model is designed to remain static, but _Licenses_ are designed to accommodate future changes -`EnterpriseSubscriptionLicenseType` and `EnterpriseSubscriptionLicense` in this PR describe only the current type of license, referred to as "classic licenses", but we can extend this in the future for e.g. new models (refreshable licenses?) or new products (Cody-only? PLG enterprise?), or existing problems (test instance licenses?) - **Granular history**: Instead of a `createdAt`, `isArchived`, `revokedAt` and and so on, the new API defines Kubernetes-style `conditions` for licenses and subscriptions to describe creation, archival, and revocation events respectively, and can be more flexibly extended for future events and a lightweight audit log of major changes to a subscription or license. In particular, `revokedAt` already has a `revokedReason` - this allows us to extend these important events with additional metadata in a flexible manner. - **Pagination**: I couldn't find a shared internal or off-the-shelf representation of pagination attributes, but each `List*` RPC describes `page_size`, `page_token`, and `next_page_token` - **Querying/filtering**: I couldn't find a strong standard for this either, but in general: - `Get*` accepts `query` that is a `oneof`, with the goal of providing exact matches only. - `List*` accepts `repeated filter`, where each `filter` is a `oneof` a set of strategies relevant to a particular `List*` RPC. Multiple filters are treated as `AND`-concatenated. Some major changes from the existing model: - **Downgrade the concept of "subscription access token"**: this was built for Cody Gateway but I am not sure it has aged well, as the mechanism is still tied to individual licenses, did not find new non-Cody-Gateway use cases (except for license checks, though those do not require an "access token" model either), and today are still not "true" access tokens as they cannot be expired/managed properly. This PR relegates the concept to remain Cody-specific as it effectively is today so that we might be able to introduce a better subscription-wide model if the use case arises. Over time, we may want to make this even more opaque, relying entirely on zero-config instead (generating from license keys). - **Subscriptions are no longer attached to a single dotcom user**: Most of these users today are not real users anyway, as our license creation process asks that you create a fake user account (["User account: [...] We create a company-level account for this."](https://handbook.sourcegraph.com/departments/technical-success/ce/process/license_keys/#license-key-mechanics)). The new API removes the concept entirely, in favour of a true user access management system in CORE-102. - **Database/GraphQL IDs** are no longer exposed - we use external, prefixed UUIDs for representing entities over APIs in a universal manner. - **Per-subscription Cody Gateway access no longer exposes `allowed models`**: I suggested this to @rafax in light of recent problems with propagating new models to Enterprise customers. He agreed that the general product direction is "model options as a selling point" - it no longer makes sense to configure these at a per-subscription level. Instead, the Cody Gateway service should configure globally allowed models directly, and each Sourcegraph instance can determine what models they trust. If we really need this back we can add it later, but for now I think this removal is the right direction. ### Direct translations `cmd/cody-gateway/internal/dotcom/operations.graphql` defines our key dependencies for achieving CORE-97. The concepts referred in `operations.graphql` translate to this new API as follows: - `dotcom { productSubscriptionByAccessToken(accessToken) }`: `codygateway.v1.GetCodyGatewayAccess({ access_token })` - `dotcom { productSubscriptions }`: `codygateway.v1.ListCodyGatewayAccess()` - `fragment ProductSubscriptionState`: - `id`: **n/a** - `uuid`: `subscriptions.v1.EnterpriseSubscription.id` - `account { username }`: `subscriptions.v1.EnterpriseSubscription.display_name` - `isArchived`: `subscriptions.v1.EnterpriseSubscription.conditions` - `codyGatewayAccess { ... }`: **separate RPC to `codygateway.v1.GetCodyGatewayAccess`** - `activeLicense { ... }`: **separate RPC to `subscriptions.v1.ListEnterpriseSubscriptionLicenses`** ### Why `lib/enterpriseportal`? We recently had to move another Telemetry Gateway to `lib`: #62061. Inevitably, there will be services that live outside the monorepo that want to integrate with Enterprise Portal (one is on our roadmap: Cody Analytics in https://github.com/sourcegraph/cody-analytics). This allows us to share generated bindings and some useful helpers, while keeping things in the monorepo. ### Implications for Cody Clients For now (and in the future), nothing is likely to change. Here's how I imagine things playing out: ```mermaid graph TD cc["Cody Clients"] -- unified API --> cg[services like Cody Gateway] cg -- PLG users --> ssc[Self-Serve Cody] cg -- Enterprise users --> ep[Enterprise Portal] ``` ## Test plan CI passes, the schemas can be generated by hand: ``` sg gen buf \ lib/enterpriseportal/subscriptions/v1/buf.gen.yaml \ lib/enterpriseportal/codyaccess/v1/buf.gen.yaml ``` --------- Co-authored-by: Joe Chen <joe@sourcegraph.com> Co-authored-by: Chris Smith <chrsmith@users.noreply.github.com>
2024-05-15 19:58:55 +00:00
connectrpc.com/connect v1.16.1
github.com/Masterminds/semver v1.5.0
github.com/charmbracelet/glamour v0.5.0
github.com/cockroachdb/errors v1.11.1
github.com/cockroachdb/redact v1.1.5
github.com/derision-test/go-mockgen/v2 v2.0.1
github.com/fatih/color v1.15.0
github.com/ghodss/yaml v1.0.0
github.com/go-enry/go-enry/v2 v2.8.8
enterprise-portal: implement basic MSP IAM and RPCs (#63173) Closes CORE-99, closes CORE-176 This PR is based off (and was also served as PoC of) [RFC 962: MSP IAM framework](https://docs.google.com/document/d/1ItJlQnpR5AHbrfAholZqjH8-8dPF1iQcKh99gE6SSjs/edit). It comes with two main parts: 1. The initial version of the MSP IAM SDK: `lib/managedservicesplatform/iam` - Embeds the [OpenFGA server implementation](https://github.com/openfga/openfga/tree/main/pkg/server) and exposes the a `ClientV1` for interacting with it. - Automagically manages the both MSP IAM's and OpenFGA's database migrations upon initializing the `ClientV1`. ![CleanShot 2024-06-18 at 15 09 24@2x](https://github.com/sourcegraph/sourcegraph/assets/2946214/387e0e28-a6c2-4664-b946-0ea4a1dd0804) - Ensures the specified OpenFGA's store and automatization model DSL exists. - Utility types and helpers to avoid easy mistakes (i.e. make the relation tuples a bit more strongly-typed). - Decided to put all types and pre-defined values together to simulate a "central registry" and acting as a forcing function for services to form some sort of convention. Then when we migrate the OpenFGA server to a separate standalone service, it will be less headache about consolidating similar meaning types/relations but different string literals. 1. The first use case of the MSP IAM: `cmd/enterprise-portal/internal/subscriptionsservice` - Added/updated RPCs: - Listing enterprise subscriptions via permissions - Update enterprise subscriptions to assign instance domains - Update enterprise subscriptions membership to assign roles (and permissions) - A database table for enterprise subscriptions, only storing the extra instance domains as Enterprise Portal is not the writeable-source-of-truth. ## Other minor changes - Moved `internal/redislock` to `lib/redislock` to be used in MSP IAM SDK. - Call `createdb ...` as part of `enterprise-portal` install script in `sg.config.yaml` (`msp_iam` database is a hard requirement of MSP IAM framework). ## Test plan Tested with gRPC UI: - `UpdateEnterpriseSubscription` to assign an instance domain - `UpdateEnterpriseSubscriptionMembership` to assign roles - `ListEnterpriseSubscriptions`: - List by subscription ID - List by instance domain - List by view cody analytics permissions --------- Co-authored-by: Robert Lin <robert@bobheadxi.dev>
2024-06-20 01:46:48 +00:00
github.com/go-redsync/redsync/v4 v4.13.0
github.com/gobwas/glob v0.2.3
github.com/google/go-cmp v0.6.0
github.com/google/uuid v1.6.0
2023-01-27 14:30:35 +00:00
github.com/grafana/regexp v0.0.0-20221123153739-15dc172cd2db
2023-02-21 09:37:13 +00:00
github.com/hexops/autogold/v2 v2.0.3
github.com/jackc/pgconn v1.14.3
github.com/json-iterator/go v1.1.12
github.com/klauspost/pgzip v1.2.5
github.com/mattn/go-isatty v0.0.18
2023-01-27 14:30:35 +00:00
github.com/mattn/go-runewidth v0.0.14
github.com/mitchellh/copystructure v1.2.0
github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6
github.com/muesli/termenv v0.12.0
enterprise-portal: implement basic MSP IAM and RPCs (#63173) Closes CORE-99, closes CORE-176 This PR is based off (and was also served as PoC of) [RFC 962: MSP IAM framework](https://docs.google.com/document/d/1ItJlQnpR5AHbrfAholZqjH8-8dPF1iQcKh99gE6SSjs/edit). It comes with two main parts: 1. The initial version of the MSP IAM SDK: `lib/managedservicesplatform/iam` - Embeds the [OpenFGA server implementation](https://github.com/openfga/openfga/tree/main/pkg/server) and exposes the a `ClientV1` for interacting with it. - Automagically manages the both MSP IAM's and OpenFGA's database migrations upon initializing the `ClientV1`. ![CleanShot 2024-06-18 at 15 09 24@2x](https://github.com/sourcegraph/sourcegraph/assets/2946214/387e0e28-a6c2-4664-b946-0ea4a1dd0804) - Ensures the specified OpenFGA's store and automatization model DSL exists. - Utility types and helpers to avoid easy mistakes (i.e. make the relation tuples a bit more strongly-typed). - Decided to put all types and pre-defined values together to simulate a "central registry" and acting as a forcing function for services to form some sort of convention. Then when we migrate the OpenFGA server to a separate standalone service, it will be less headache about consolidating similar meaning types/relations but different string literals. 1. The first use case of the MSP IAM: `cmd/enterprise-portal/internal/subscriptionsservice` - Added/updated RPCs: - Listing enterprise subscriptions via permissions - Update enterprise subscriptions to assign instance domains - Update enterprise subscriptions membership to assign roles (and permissions) - A database table for enterprise subscriptions, only storing the extra instance domains as Enterprise Portal is not the writeable-source-of-truth. ## Other minor changes - Moved `internal/redislock` to `lib/redislock` to be used in MSP IAM SDK. - Call `createdb ...` as part of `enterprise-portal` install script in `sg.config.yaml` (`msp_iam` database is a hard requirement of MSP IAM framework). ## Test plan Tested with gRPC UI: - `UpdateEnterpriseSubscription` to assign an instance domain - `UpdateEnterpriseSubscriptionMembership` to assign roles - `ListEnterpriseSubscriptions`: - List by subscription ID - List by instance domain - List by view cody analytics permissions --------- Co-authored-by: Robert Lin <robert@bobheadxi.dev>
2024-06-20 01:46:48 +00:00
github.com/redis/go-redis/v9 v9.5.3
2024-01-08 19:06:01 +00:00
github.com/sourcegraph/conc v0.3.1-0.20240108182409-4afefce20f9b
2023-01-27 14:30:35 +00:00
github.com/sourcegraph/go-diff v0.6.2-0.20221123165719-f8cd299c40f3
github.com/sourcegraph/jsonx v0.0.0-20200629203448-1a936bd500cf
github.com/sourcegraph/log v0.0.0-20231018134238-fbadff7458bb
2023-06-28 18:06:11 +00:00
github.com/sourcegraph/scip v0.3.1-0.20230627154934-45df7f6d33fc
github.com/stretchr/testify v1.8.4
github.com/urfave/cli/v2 v2.23.7
github.com/xeipuuv/gojsonschema v1.2.0
golang.org/x/oauth2 v0.21.0
golang.org/x/sys v0.21.0
golang.org/x/term v0.21.0
google.golang.org/grpc v1.65.0
google.golang.org/protobuf v1.34.1
gopkg.in/yaml.v2 v2.4.0
gopkg.in/yaml.v3 v3.0.1
pgregory.net/rapid v1.1.0
)
require (
google.golang.org/genproto/googleapis/api v0.0.0-20240528184218-531527333157 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240528184218-531527333157 // indirect
)
require (
cloud.google.com/go/compute/metadata v0.3.0 // indirect
github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 // indirect
2023-01-27 14:30:35 +00:00
github.com/Masterminds/goutils v1.1.1 // indirect
github.com/Masterminds/sprig v2.22.0+incompatible // indirect
github.com/alecthomas/chroma v0.10.0 // indirect
github.com/alecthomas/kingpin v2.2.6+incompatible // indirect
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751 // indirect
github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137 // indirect
github.com/aymerick/douceur v0.2.0 // indirect
github.com/bufbuild/buf v1.4.0 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/cockroachdb/logtags v0.0.0-20230118201751-21c54148d20b // indirect
github.com/cpuguy83/go-md2man/v2 v2.0.2 // indirect
2022-10-26 14:07:29 +00:00
github.com/dave/jennifer v1.5.0 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
enterprise-portal: implement basic MSP IAM and RPCs (#63173) Closes CORE-99, closes CORE-176 This PR is based off (and was also served as PoC of) [RFC 962: MSP IAM framework](https://docs.google.com/document/d/1ItJlQnpR5AHbrfAholZqjH8-8dPF1iQcKh99gE6SSjs/edit). It comes with two main parts: 1. The initial version of the MSP IAM SDK: `lib/managedservicesplatform/iam` - Embeds the [OpenFGA server implementation](https://github.com/openfga/openfga/tree/main/pkg/server) and exposes the a `ClientV1` for interacting with it. - Automagically manages the both MSP IAM's and OpenFGA's database migrations upon initializing the `ClientV1`. ![CleanShot 2024-06-18 at 15 09 24@2x](https://github.com/sourcegraph/sourcegraph/assets/2946214/387e0e28-a6c2-4664-b946-0ea4a1dd0804) - Ensures the specified OpenFGA's store and automatization model DSL exists. - Utility types and helpers to avoid easy mistakes (i.e. make the relation tuples a bit more strongly-typed). - Decided to put all types and pre-defined values together to simulate a "central registry" and acting as a forcing function for services to form some sort of convention. Then when we migrate the OpenFGA server to a separate standalone service, it will be less headache about consolidating similar meaning types/relations but different string literals. 1. The first use case of the MSP IAM: `cmd/enterprise-portal/internal/subscriptionsservice` - Added/updated RPCs: - Listing enterprise subscriptions via permissions - Update enterprise subscriptions to assign instance domains - Update enterprise subscriptions membership to assign roles (and permissions) - A database table for enterprise subscriptions, only storing the extra instance domains as Enterprise Portal is not the writeable-source-of-truth. ## Other minor changes - Moved `internal/redislock` to `lib/redislock` to be used in MSP IAM SDK. - Call `createdb ...` as part of `enterprise-portal` install script in `sg.config.yaml` (`msp_iam` database is a hard requirement of MSP IAM framework). ## Test plan Tested with gRPC UI: - `UpdateEnterpriseSubscription` to assign an instance domain - `UpdateEnterpriseSubscriptionMembership` to assign roles - `ListEnterpriseSubscriptions`: - List by subscription ID - List by instance domain - List by view cody analytics permissions --------- Co-authored-by: Robert Lin <robert@bobheadxi.dev>
2024-06-20 01:46:48 +00:00
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
2023-01-27 14:30:35 +00:00
github.com/dlclark/regexp2 v1.7.0 // indirect
github.com/dustin/go-humanize v1.0.0 // indirect
github.com/envoyproxy/protoc-gen-validate v1.0.4 // indirect
github.com/getsentry/sentry-go v0.25.0 // indirect
github.com/go-enry/go-oniguruma v1.2.1 // indirect
github.com/gofrs/flock v0.8.1 // indirect
github.com/gofrs/uuid v4.2.0+incompatible // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/google/gofuzz v1.2.0 // indirect
github.com/gorilla/css v1.0.0 // indirect
enterprise-portal: implement basic MSP IAM and RPCs (#63173) Closes CORE-99, closes CORE-176 This PR is based off (and was also served as PoC of) [RFC 962: MSP IAM framework](https://docs.google.com/document/d/1ItJlQnpR5AHbrfAholZqjH8-8dPF1iQcKh99gE6SSjs/edit). It comes with two main parts: 1. The initial version of the MSP IAM SDK: `lib/managedservicesplatform/iam` - Embeds the [OpenFGA server implementation](https://github.com/openfga/openfga/tree/main/pkg/server) and exposes the a `ClientV1` for interacting with it. - Automagically manages the both MSP IAM's and OpenFGA's database migrations upon initializing the `ClientV1`. ![CleanShot 2024-06-18 at 15 09 24@2x](https://github.com/sourcegraph/sourcegraph/assets/2946214/387e0e28-a6c2-4664-b946-0ea4a1dd0804) - Ensures the specified OpenFGA's store and automatization model DSL exists. - Utility types and helpers to avoid easy mistakes (i.e. make the relation tuples a bit more strongly-typed). - Decided to put all types and pre-defined values together to simulate a "central registry" and acting as a forcing function for services to form some sort of convention. Then when we migrate the OpenFGA server to a separate standalone service, it will be less headache about consolidating similar meaning types/relations but different string literals. 1. The first use case of the MSP IAM: `cmd/enterprise-portal/internal/subscriptionsservice` - Added/updated RPCs: - Listing enterprise subscriptions via permissions - Update enterprise subscriptions to assign instance domains - Update enterprise subscriptions membership to assign roles (and permissions) - A database table for enterprise subscriptions, only storing the extra instance domains as Enterprise Portal is not the writeable-source-of-truth. ## Other minor changes - Moved `internal/redislock` to `lib/redislock` to be used in MSP IAM SDK. - Call `createdb ...` as part of `enterprise-portal` install script in `sg.config.yaml` (`msp_iam` database is a hard requirement of MSP IAM framework). ## Test plan Tested with gRPC UI: - `UpdateEnterpriseSubscription` to assign an instance domain - `UpdateEnterpriseSubscriptionMembership` to assign roles - `ListEnterpriseSubscriptions`: - List by subscription ID - List by instance domain - List by view cody analytics permissions --------- Co-authored-by: Robert Lin <robert@bobheadxi.dev>
2024-06-20 01:46:48 +00:00
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/hexops/gotextdiff v1.0.3 // indirect
2023-02-21 09:37:13 +00:00
github.com/hexops/valast v1.4.3 // indirect
2023-01-27 14:30:35 +00:00
github.com/huandu/xstrings v1.3.2 // indirect
github.com/imdario/mergo v0.3.13 // indirect
github.com/inconshreveable/mousetrap v1.0.0 // indirect
github.com/jackc/chunkreader/v2 v2.0.1 // indirect
github.com/jackc/pgio v1.0.0 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgproto3/v2 v2.3.3 // indirect
dev/msp: add CloudSQL integration (#58420) This PR has 3 major changes: 1. Provisioning a Cloud SQL PostgreSQL instance: `resource.postgreSQL` 2. Updates to the MSP runtime for connecting to the database 3. Publishable `msp-example` for the `msp-testbed` deployment, for testing integrations and other MSP features, and adds an example Gorm setup Closes https://github.com/sourcegraph/sourcegraph/issues/56848. For more context, see go/msp ### `resource.postgreSQL` This change adds `resource.postgreSQL` to the spec for provisioning a Cloud SQL database. Example: ```yaml resources: postgreSQL: databases: ["primary"] cpu: 1 memoryGB: 4 ``` When configured, a Cloud SQL instance is provisioned, and the Cloud Run workload SA is granted access to it. The specified `databases` are also created. `cpu` and `memoryGB` are used to generate the custom VM to provision the instance with. A lot of the base configuration for the database instance are taken directly from what we use in Cloud: https://github.com/sourcegraph/controller/blob/9cac1b1aa0dde038280d424f4d4b6c34ffb22e36/internal/resource/sql/sql.go#L116, and is similar to the Redis setup, in that we need the private peering VPC and install additional certs on the Cloud Run instance. ### MSP runtime `service.Contract` now has a helper, `GetPostgreSQLDB(database)`, for generating the appropriate configuration for creating a connection to a given database and returning a `sql.DB`, which most libraries have adapters for. The helper sets up `pgx.ConnConfig` that uses workload identity to connect directly to Cloud SQL, following the guide here: https://github.com/GoogleCloudPlatform/cloud-sql-go-connector?tab=readme-ov-file#automatic-iam-database-authentication In local dev, `PGDSN` can be used to hardcode an appropriate value. ### `msp-example` I've moved the MSP example from #56846 to the top-level `cmd/msp-example,` and set up some guidance for publishing it. This service now also includes a simple database connection and setup using Gorm, used by https://github.com/sourcegraph/accounts.sourcegraph.com, and can be used for testing, e.g. https://github.com/sourcegraph/managed-services/pull/87 ## Test plan - [x] Deploy https://github.com/sourcegraph/managed-services/pull/87, which will connect to the database and set up some tables - [x] Review diff: https://github.com/sourcegraph/managed-services/actions/runs/6951246277 - [x] Tear down database in https://github.com/sourcegraph/managed-services/pull/87, with caveat: requires state surgery. Should be rare enough (we might prefer to tear down environments entirely) - see docstrings in package `postgresqlroles` --------- Co-authored-by: Michael Lin <mlzc@hey.com> Co-authored-by: Joe Chen <joe@sourcegraph.com>
2023-11-22 23:32:53 +00:00
github.com/jackc/pgservicefile v0.0.0-20221227161230-091c0ba34f0a // indirect
github.com/jdxcode/netrc v0.0.0-20210204082910-926c7f70242a // indirect
github.com/jhump/protocompile v0.0.0-20220216033700-d705409f108f // indirect
github.com/jhump/protoreflect v1.12.1-0.20220417024638-438db461d753 // indirect
github.com/klauspost/compress v1.16.0 // indirect
2023-01-27 14:30:35 +00:00
github.com/kr/pretty v0.3.1 // indirect
github.com/kr/text v0.2.0 // indirect
github.com/lucasb-eyer/go-colorful v1.2.0 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/microcosm-cc/bluemonday v1.0.23 // indirect
github.com/mitchellh/go-wordwrap v1.0.1 // indirect
github.com/mitchellh/reflectwalk v1.0.2 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/muesli/reflow v0.3.0 // indirect
2023-01-27 14:30:35 +00:00
github.com/mwitkow/go-proto-validators v0.3.2 // indirect
github.com/nightlyone/lockfile v1.0.0 // indirect
github.com/olekukonko/tablewriter v0.0.5 // indirect
github.com/pkg/browser v0.0.0-20210911075715-681adbf594b8 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pkg/profile v1.6.0 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/pseudomuto/protoc-gen-doc v1.5.1 // indirect
github.com/pseudomuto/protokit v0.2.0 // indirect
2023-01-27 14:30:35 +00:00
github.com/rivo/uniseg v0.4.2 // indirect
github.com/rogpeppe/go-internal v1.11.0 // indirect
github.com/russross/blackfriday/v2 v2.1.0 // indirect
2023-01-27 14:30:35 +00:00
github.com/spf13/cobra v1.5.0 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb // indirect
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect
github.com/xrash/smetrics v0.0.0-20201216005158-039620a65673 // indirect
2023-01-27 14:30:35 +00:00
github.com/yuin/goldmark v1.5.2 // indirect
github.com/yuin/goldmark-emoji v1.0.1 // indirect
2023-01-27 14:30:35 +00:00
go.opencensus.io v0.24.0 // indirect
go.uber.org/atomic v1.11.0 // indirect
2023-01-27 14:30:35 +00:00
go.uber.org/goleak v1.2.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
2023-01-27 14:30:35 +00:00
go.uber.org/zap v1.24.0 // indirect
golang.org/x/crypto v0.24.0 // indirect
golang.org/x/mod v0.17.0 // indirect
golang.org/x/net v0.26.0 // indirect
golang.org/x/sync v0.7.0 // indirect
golang.org/x/text v0.16.0 // indirect
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d // indirect
dev/msp: add CloudSQL integration (#58420) This PR has 3 major changes: 1. Provisioning a Cloud SQL PostgreSQL instance: `resource.postgreSQL` 2. Updates to the MSP runtime for connecting to the database 3. Publishable `msp-example` for the `msp-testbed` deployment, for testing integrations and other MSP features, and adds an example Gorm setup Closes https://github.com/sourcegraph/sourcegraph/issues/56848. For more context, see go/msp ### `resource.postgreSQL` This change adds `resource.postgreSQL` to the spec for provisioning a Cloud SQL database. Example: ```yaml resources: postgreSQL: databases: ["primary"] cpu: 1 memoryGB: 4 ``` When configured, a Cloud SQL instance is provisioned, and the Cloud Run workload SA is granted access to it. The specified `databases` are also created. `cpu` and `memoryGB` are used to generate the custom VM to provision the instance with. A lot of the base configuration for the database instance are taken directly from what we use in Cloud: https://github.com/sourcegraph/controller/blob/9cac1b1aa0dde038280d424f4d4b6c34ffb22e36/internal/resource/sql/sql.go#L116, and is similar to the Redis setup, in that we need the private peering VPC and install additional certs on the Cloud Run instance. ### MSP runtime `service.Contract` now has a helper, `GetPostgreSQLDB(database)`, for generating the appropriate configuration for creating a connection to a given database and returning a `sql.DB`, which most libraries have adapters for. The helper sets up `pgx.ConnConfig` that uses workload identity to connect directly to Cloud SQL, following the guide here: https://github.com/GoogleCloudPlatform/cloud-sql-go-connector?tab=readme-ov-file#automatic-iam-database-authentication In local dev, `PGDSN` can be used to hardcode an appropriate value. ### `msp-example` I've moved the MSP example from #56846 to the top-level `cmd/msp-example,` and set up some guidance for publishing it. This service now also includes a simple database connection and setup using Gorm, used by https://github.com/sourcegraph/accounts.sourcegraph.com, and can be used for testing, e.g. https://github.com/sourcegraph/managed-services/pull/87 ## Test plan - [x] Deploy https://github.com/sourcegraph/managed-services/pull/87, which will connect to the database and set up some tables - [x] Review diff: https://github.com/sourcegraph/managed-services/actions/runs/6951246277 - [x] Tear down database in https://github.com/sourcegraph/managed-services/pull/87, with caveat: requires state surgery. Should be rare enough (we might prefer to tear down environments entirely) - see docstrings in package `postgresqlroles` --------- Co-authored-by: Michael Lin <mlzc@hey.com> Co-authored-by: Joe Chen <joe@sourcegraph.com>
2023-11-22 23:32:53 +00:00
google.golang.org/genproto v0.0.0-20231030173426-d783a09b4405 // indirect
2023-01-27 14:30:35 +00:00
gotest.tools/v3 v3.0.3 // indirect
2023-02-21 09:37:13 +00:00
mvdan.cc/gofumpt v0.4.0 // indirect
)
// See: https://github.com/ghodss/yaml/pull/65
replace github.com/ghodss/yaml => github.com/sourcegraph/yaml v1.0.1-0.20200714132230-56936252f152