In the upcoming release, we will only support gRPC going forward. This PR removes the old HTTP client and server implementations and a few leftovers from the transition.
This PR ships our freshly rewritten container images built with
rules_oci and Wolfi, which for now will only be used on S2.
*What is this about*
This work is the conjunction of [hardening container
images](https://github.com/orgs/sourcegraph/projects/302?pane=issue&itemId=25019223)
and fully building our container images with Bazel.
* All base images are now distroless, based on Wolfi, meaning we fully
control every little package version and we won't be subject anymore to
Alpine maintainers dropping a postgres version for example.
* Container images are now built with `rules_oci`, meaning we don't have
Dockerfile anymore, but instead created through [Bazel
rules](https://sourcegraph.sourcegraph.com/github.com/sourcegraph/sourcegraph@bzl/oci_wolfi/-/blob/enterprise/cmd/gitserver/BUILD.bazel).
Don't be scared, while this will look a bit strange to you at first,
it's much saner and simpler to do than our Dockerfiles and their muddy
shell scripts calling themselves in cascade.
:spiral_note_pad: *Plan*:
*1/ (NOW) We merge our branch on `main` today, here is what it does
change for you 👇:skin-tone-3::*
* On `main`:
* It will introduce a new job on `main` _Bazel Push_, which will push
those new images on our registries with all tags prefixed by `bazel-`.
* These new images will be picked up by S2 and S2 only.
* The existing jobs building docker images and pushing them will stay in
place until we have QA'ed them enough and are confident to roll them out
on Dotcom.
* Because we'll be building both images, there will be more jobs running
on `main`, but this should not affect the wall clock time.
* On all branches (so your PRs and `main`)
* The _Bazel Test_ job will now run: Backend Integration Tests, E2E
Tests and CodeIntel QA
* This will increase the duration of your test jobs in PRs, but as we
haven't removed yet the `sg lint` step, it should not affect too much
the wall clock time of your PRs.
* But it will also increase your confidence toward your changes, as the
coverage will vastly increased compared to before.
* If you have ongoing branches which are affecting the docker images
(like adding a new binary, like the recent `scip-tags`, reach us out on
#job-fair-bazel so we can help you to port your changes. It's much much
simpler than before, but it's going to be unfamiliar to you).
* If something goes awfully wrong, we'll rollback and update this
thread.
*2/ (EOW / Early next week) Once we're confident enough with what we saw
on S2, we'll roll the new images on Dotcom.*
* After the first successful deploy and a few sanity checks, we will
drop the old images building jobs.
* At this point, we'll reach out to all TLs asking for their help to
exercise all features of our product to ensure we catch any potential
breakage.
## Test plan
<!-- All pull requests REQUIRE a test plan:
https://docs.sourcegraph.com/dev/background-information/testing_principles
-->
* We tested our new images on `scale-testing` and it worked.
* The new container building rules comes with _container tests_ which
ensures that produced images are containing and configured with what
should be in there:
[example](https://sourcegraph.sourcegraph.com/github.com/sourcegraph/sourcegraph@bzl/oci_wolfi/-/blob/enterprise/cmd/gitserver/image_test.yaml)
.
---------
Co-authored-by: Dave Try <davetry@gmail.com>
Co-authored-by: Will Dollman <will.dollman@sourcegraph.com>
- Run client E2E tests in Bazel
- Fix client integration tests execution locally
- Make client integration tests work with different
`SOURCEGRAPH_BASE_URL` values
- Clean up integration tests and E2E tests logs
## Test plan
1. CI
2. `bazel test //client/web/src/integration:integration-tests`
3. `bazel test //client/web/src/end-to-end:e2e`
---------
Co-authored-by: Jason Bedard <jason@aspect.dev>
Co-authored-by: William Bezuidenhout <william.bezuidenhout@sourcegraph.com>
Co-authored-by: Jean-Hadrien Chabran <jh@chabran.fr>
Co-authored-by: davejrt <davetry@gmail.com>
The migration to gRPC will take a significant amount of time, and until
we are fully migrated and are able to remove the non-gRPC codepaths, we
need to be confident that both codepaths work correctly.
Our options here include:
1) duplicating every test to run with and without gRPC
2) adding a `TestMain` to every test package that reruns the suite with
and without gRPC
3) manually adding tests for all the codepaths that are interesting to
gRPC
4) always running both the HTTP and gRPC codepaths in parallel
5) mirror the test steps in CI to run both with and without gRPC
None of the options are good, but a solution here is necessary avoid
introducing regressions during the migration. This PR implements option
5 because it is the least bad option that is also the easiest to rip out
when it is no longer needed.
This PR updates our CI pipeline generator to duplicate most of our go
tests (everything except database tests) as well as our backend
integration tests. It does this by adding the environment variable
`SG_FEATURE_FLAG_GRPC`, which is respected when clients are deciding
whether to use gRPC or HTTP.
Co-authored-by: Geoffrey Gilmore <me@ggilmore.net>
* local install code insight landing page & hide code insight action when feature not avail
* Fix autocorrect
* Missing dep
* Use window.context.enabledCodeInsights
- unify `sourcegraph/server` setup in integration tests so that we use the same scripts in each
- move all integration tests into new `dev/ci/integration` directory, and introduce a structure of `run.sh` and `test.sh` - see `dev/ci/integration/README.md` for overview
- some minor refactors to unify the tests a bit more
- remove unused `servers.yaml` that was part of the Vagrant setup
* use .Critical accessors; use conf.Unified type
* pkg/conf: add unified configuration types
* {pkg/conf,cmd/...}: rewrite conf package to be database-backed
* pkg/legacyconf: import from sourcegraph@v2.13.6 pkg/conf/ (and reduce)
* legacyschema: import from sourcegraph@v2.13.6 schema/ (and reduce)
* add dev config override support
* cmd/management-console/assets: add assets package for packing web app code
* cmd/management-console/web: initial webapp implementation
* schema: split site.schema.json into site and critical configuration portions
* schema: critical: add gitlab as valid auth.providers key (broken in master)
* web: update to reflect site config changes
* cmd/management-console: initial backend implementation
* cmd/management-console: add Go backend
* cmd/management-console: add TS frontend
* web: site-admin: add management console password alert
* cmd/management-console/internal/tlscertgen: package for TLS certificate generation
* schema: update docs
* upgrade deasync
* pkg/conf: add proper config defaults for each deployment type
* vfsgendev installation fix
* mgmt console web package-lock.json change
* assets doc.go + gitignore change
* gofmt assets
* dev/check: use dev build tag
Some things such as assets are only defined behind a dev build tag OR
after generating something via `go generate`. Since go generate has not
run yet, I am opting to use the `dev` build tag here.
* cmd/management-console/auth_test.go: simplify test
* fix go assets
* pkg/conf/confdefaults
* pkg/conf: validation test fix
* web: update to reflect site configuration changes
* expose management console port in docker run commands
* management-console: go: return concrete type to /update requests, not type that can arbitrarily change in future
* management-console: web: fix saving ID bug + properly display errors
* pkg/db/confdb: return an error if the creator is not up to date
* web mgmt console error handling
* mgmt console go backend ErrNewerEdit
* pkg/conf: fix zero configuration check (marshaled JSON is "{}")
* add linter for accidental transitive imports of pkg/conf in mgmt console
* enterprise/dev/ci: fix building of non-enterprise docker-images branches
* NOCHANGELOG
