Commit Graph

2 Commits

Author SHA1 Message Date
Noah S-C
98e0f75d1e
bazel: use transitions to apply cross-compile platform automatically to oci_image (#60569)
Removes the need to pass `--config=docker-darwin` through the following mechanisms:

1. `--enable_platform_specific_config` to enable certain flags on macos only e.g. `--extra_toolchains @zig_sdk//toolchain:linux_amd64_gnu.2.34` and `--sandbox_add_mount_pair=/tmp` (see [.bazelrc change](https://github.com/sourcegraph/sourcegraph/pull/60569/files?file-filters%5B%5D=dotfile&show-viewed-files=true))
2. Apply a transition (using https://github.com/fmeum/with_cfg.bzl, please view [the following great video on it](https://www.youtube.com/watch?v=U5bdQRQY-io)) on `oci_image` targets when on the `@platforms//os:macos` platform to transition to the `@zig_sdk//platform:linux_amd64` platform. 
	- This will start at `oci_image` targets and propagate down to e.g. `go_{binary,library}` etc targets with the "transitioned" platform configuration, resulting in them being built with the transitioned-to platform
3. Remove `darwin_docker_e2e_go` config_setting and `darwin-docker` bool_flag.
	- These aren't necessary anymore, as the places where these were used were not in the transitive closure rooted at an `oci_image` target, meaning they wouldn't be transitioned.

To review, view [the following (filtered) files](https://github.com/sourcegraph/sourcegraph/pull/60569/files?file-filters%5B%5D=.bzl&file-filters%5B%5D=.sh&file-filters%5B%5D=.yaml&file-filters%5B%5D=No+extension&file-filters%5B%5D=dotfile&show-viewed-files=true)  along with [the root BUILD.bazel](https://github.com/sourcegraph/sourcegraph/pull/60569/files#diff-7fc57714ef13c3325ce2a1130202edced92fcccc0c6db34a72f7b57f60d552a3). All the other files are just changing the `load` statements from `@rules_oci` to `//dev:oci_defs.bzl`

## Test plan

CI, checked image locally and `sg test bazel-backend-integration` & `sg test bazel-e2e`
2024-02-20 13:57:56 +00:00
Robert Lin
1f85781471
dev/msp: add CloudSQL integration (#58420)
This PR has 3 major changes:

1. Provisioning a Cloud SQL PostgreSQL instance: `resource.postgreSQL`
2. Updates to the MSP runtime for connecting to the database
3. Publishable `msp-example` for the `msp-testbed` deployment, for testing integrations and other MSP features, and adds an example Gorm setup

Closes https://github.com/sourcegraph/sourcegraph/issues/56848. For more context, see go/msp

### `resource.postgreSQL`

This change adds `resource.postgreSQL` to the spec for provisioning a Cloud SQL database. Example:

```yaml
  resources:
    postgreSQL:
      databases: ["primary"]
      cpu: 1
      memoryGB: 4
```

When configured, a Cloud SQL instance is provisioned, and the Cloud Run workload SA is granted access to it. The specified `databases` are also created. `cpu` and `memoryGB` are used to generate the custom VM to provision the instance with. A lot of the base configuration for the database instance are taken directly from what we use in Cloud: 9cac1b1aa0/internal/resource/sql/sql.go (L116), and is similar to the Redis setup, in that we need the private peering VPC and install additional certs on the Cloud Run instance.

### MSP runtime

`service.Contract` now has a helper, `GetPostgreSQLDB(database)`, for generating the appropriate configuration for creating a connection to a given database and returning a `sql.DB`, which most libraries have adapters for. The helper sets up `pgx.ConnConfig` that uses workload identity to connect directly to Cloud SQL, following the guide here: https://github.com/GoogleCloudPlatform/cloud-sql-go-connector?tab=readme-ov-file#automatic-iam-database-authentication

In local dev, `PGDSN` can be used to hardcode an appropriate value.

### `msp-example`

I've moved the MSP example from #56846 to the top-level `cmd/msp-example,` and set up some guidance for publishing it. This service now also includes a simple database connection and setup using Gorm, used by https://github.com/sourcegraph/accounts.sourcegraph.com, and can be used for testing, e.g. https://github.com/sourcegraph/managed-services/pull/87 

## Test plan

- [x] Deploy https://github.com/sourcegraph/managed-services/pull/87, which will connect to the database and set up some tables
- [x] Review diff: https://github.com/sourcegraph/managed-services/actions/runs/6951246277
- [x] Tear down database in https://github.com/sourcegraph/managed-services/pull/87, with caveat: requires state surgery. Should be rare enough (we might prefer to tear down environments entirely) - see docstrings in package `postgresqlroles`

---------

Co-authored-by: Michael Lin <mlzc@hey.com>
Co-authored-by: Joe Chen <joe@sourcegraph.com>
2023-11-22 15:32:53 -08:00