mirror of
https://github.com/sourcegraph/sourcegraph.git
synced 2026-02-06 20:51:43 +00:00
58da6780d7
This PR ships our freshly rewritten container images built with rules_oci and Wolfi, which for now will only be used on S2. *What is this about* This work is the conjunction of [hardening container images](https://github.com/orgs/sourcegraph/projects/302?pane=issue&itemId=25019223) and fully building our container images with Bazel. * All base images are now distroless, based on Wolfi, meaning we fully control every little package version and we won't be subject anymore to Alpine maintainers dropping a postgres version for example. * Container images are now built with `rules_oci`, meaning we don't have Dockerfile anymore, but instead created through [Bazel rules](https://sourcegraph.sourcegraph.com/github.com/sourcegraph/sourcegraph@bzl/oci_wolfi/-/blob/enterprise/cmd/gitserver/BUILD.bazel). Don't be scared, while this will look a bit strange to you at first, it's much saner and simpler to do than our Dockerfiles and their muddy shell scripts calling themselves in cascade. :spiral_note_pad: *Plan*: *1/ (NOW) We merge our branch on `main` today, here is what it does change for you 👇:skin-tone-3::* * On `main`: * It will introduce a new job on `main` _Bazel Push_, which will push those new images on our registries with all tags prefixed by `bazel-`. * These new images will be picked up by S2 and S2 only. * The existing jobs building docker images and pushing them will stay in place until we have QA'ed them enough and are confident to roll them out on Dotcom. * Because we'll be building both images, there will be more jobs running on `main`, but this should not affect the wall clock time. * On all branches (so your PRs and `main`) * The _Bazel Test_ job will now run: Backend Integration Tests, E2E Tests and CodeIntel QA * This will increase the duration of your test jobs in PRs, but as we haven't removed yet the `sg lint` step, it should not affect too much the wall clock time of your PRs. * But it will also increase your confidence toward your changes, as the coverage will vastly increased compared to before. * If you have ongoing branches which are affecting the docker images (like adding a new binary, like the recent `scip-tags`, reach us out on #job-fair-bazel so we can help you to port your changes. It's much much simpler than before, but it's going to be unfamiliar to you). * If something goes awfully wrong, we'll rollback and update this thread. *2/ (EOW / Early next week) Once we're confident enough with what we saw on S2, we'll roll the new images on Dotcom.* * After the first successful deploy and a few sanity checks, we will drop the old images building jobs. * At this point, we'll reach out to all TLs asking for their help to exercise all features of our product to ensure we catch any potential breakage. ## Test plan <!-- All pull requests REQUIRE a test plan: https://docs.sourcegraph.com/dev/background-information/testing_principles --> * We tested our new images on `scale-testing` and it worked. * The new container building rules comes with _container tests_ which ensures that produced images are containing and configured with what should be in there: [example](https://sourcegraph.sourcegraph.com/github.com/sourcegraph/sourcegraph@bzl/oci_wolfi/-/blob/enterprise/cmd/gitserver/image_test.yaml) . --------- Co-authored-by: Dave Try <davetry@gmail.com> Co-authored-by: Will Dollman <will.dollman@sourcegraph.com> |
||
|---|---|---|
| .. | ||
| cmd | ||
| internal | ||
| .gitignore | ||
| CODENOTIFY | ||
| README.md | ||
Precise code intel tester
This package provides integration and load testing utilities for precise code intel services.
Prerequisites
Ensure that the following tools are available on your path:
You should have enviornment variables that authenticate you to the sourcegraph-dev GCS project if you plan to upload or download index files (as we do in CI).
Set:
SOURCEGRAPH_BASE_URL=http://localhost:3080
SOURCEGRAPH_SUDO_TOKEN=<YOUR SOURCEGRAPH API ACCESS TOKEN>
Testing
- Ensure these repositories exist on your instance (in
Site Admin->Manage code hosts->GitHub):
"repos": [
"sourcegraph-testing/etcd",
"sourcegraph-testing/tidb",
"sourcegraph-testing/titan",
"sourcegraph-testing/zap",
"sourcegraph-testing/nacelle",
"sourcegraph-testing/nacelle-config",
"sourcegraph-testing/nacelle-service",
"sourcegraph/code-intel-extensions"
],
- Download the test indexes by running the following command:
go run ./cmd/download
Alternatively, generate them by running the following command (this takes much longer):
go run ./cmd/clone-and-index
If there is previous upload or index state on the target instance, they can be cleared by running the following command:
go run ./cmd/clear
Upload the indexes to the target instance by running the following command:
go run ./cmd/upload
Then run test queries against the target instance by running the following command:
go run ./cmd/query
Refreshing indexes
If there is a change to an indexer that needs to be tested, the indexes can be regenerated and uploaded to gcloud for future test runs.
Generate indexes by running the following command:
go run ./cmd/clone-and-index
Upload the generated indexes to GCS by running the following command:
go run ./cmd/upload-gcs
Or if you just want to test an indexer change locally, you can:
rm -rf testdata/indexes/
Then run the clone-and-index step described above.