Another step towards https://github.com/sourcegraph/sourcegraph/issues/59155, previously `bazel test //...` would error at analysis time on `//client/web/src/end-to-end:e2e` due to it attempting to perform variable substitution for env vars e.g. `"HEADLESS": "$(E2E_HEADLESS)"`, for values not defined via `--define` (we only set these explicitly in .aspect/bazelrc/ci.sourcegraph.bazelrc and some `sg` targets).
By leveraging https://bazel.build/rules/lib/builtins/actions#run.use_default_shell_env, we can allow the test to read values from `--action_env` while _also_ having explicit values set via `env` macro parameter. Previously, setting `env` macro parameter would completely shadow any `--action_env` values.
Unfortunately, we cant use `--test_env` for this, as `js_run_binary` is an action not a test (or something like that?).
We also cant do env renaming anymore, meaning we have to drop the `E2E_` prefix for some env vars. At least one script needed some reworking to accommodate that `e2e_test.sh`
## Test plan
👁️ CI once again 👁️
In the upcoming release, we will only support gRPC going forward. This PR removes the old HTTP client and server implementations and a few leftovers from the transition.
* add initial aspect workflow yaml
- try reading docker config env var
- bump both timers
- bump grpc test timout to long
- skip additional perforce test and run all tests
- bump timeouts
- more timeout bumps and skip p4 test
- bump doc:test timeout
- bump e2e_test timeout
- bump database/connections/live timeout
- tag integration tests as exclusive
* add recommended bazelrc in workflows to speed up cold builds
* disable experimental_fetch_all_coverage_outputs
* port changes from https://github.com/sourcegraph/sourcegraph/compare/aspect-trial/wb-add-initial-config...aspect-trial/wb-add-initial-config-greg
* bazel configure
* add //:postcss_config_js as data target to client/web
* remove postcss added in debug
* use node-fetch and only test codeintellify
* use testing fetch.js setup
* fix syntax in testSetup
* various fixes
revert timeout bump on repository test
re-enable git p4 test
add testing from shared deps
bazel configure
* update comments on skipped tests
* restore `is_percy_enabled` for mocha_test
* slightly increase repo cloning wait
* use process.cwd instead of __dirname
* set sizing to moderate as well for embeddings
* remove setting CI in workflows yaml
* fix sizing
* workflow yaml tweaks and bazelrc tweaks
* make bazelrc consistent with what was in workflow yaml
---------
Co-authored-by: Jean-Hadrien Chabran <jh@chabran.fr>
Co-authored-by: Greg Magolan <greg@aspect.dev>
We recently refactored how to run server integration tests with Bazel,
so they are standalone tasks requiring no prior setup to run them.
This PR takes this further, by providing a macro named
`server_integration_test` to be used in `testing/BUILD.bazel` to
centralize all test runners/wrapper to run those tests who are targeting
the server image.
It also adds a how-to describing how they work at high level as well as
step-by-step guide to help teammates to start writing or maintaining the
existing scripts.
@peterguy I'm tagging you here, as per this
[message](https://sourcegraph.slack.com/archives/C04MYFW01NV/p1686850329953539?thread_ts=1686661082.758009&cid=C04MYFW01NV)
where it looked you needed to do exactly this. This makes it sound
you're the perfect reviewer for this bit of documentation.
## Test plan
<!-- All pull requests REQUIRE a test plan:
https://docs.sourcegraph.com/dev/background-information/testing_principles
-->
CI + local preview for the docs.
---------
Co-authored-by: Dave Try <davetry@gmail.com>
Changes interpreter in shell scripts to not hard-code bash location, for
portability
## Test plan
Previously `DOCKER_BAZEL=true IMAGE=strum355/frontend VERSION=5.0.5 ./enterprise/cmd/frontend/build.sh` wouldn't work for me, now it does : )
This PR ships our freshly rewritten container images built with
rules_oci and Wolfi, which for now will only be used on S2.
*What is this about*
This work is the conjunction of [hardening container
images](https://github.com/orgs/sourcegraph/projects/302?pane=issue&itemId=25019223)
and fully building our container images with Bazel.
* All base images are now distroless, based on Wolfi, meaning we fully
control every little package version and we won't be subject anymore to
Alpine maintainers dropping a postgres version for example.
* Container images are now built with `rules_oci`, meaning we don't have
Dockerfile anymore, but instead created through [Bazel
rules](https://sourcegraph.sourcegraph.com/github.com/sourcegraph/sourcegraph@bzl/oci_wolfi/-/blob/enterprise/cmd/gitserver/BUILD.bazel).
Don't be scared, while this will look a bit strange to you at first,
it's much saner and simpler to do than our Dockerfiles and their muddy
shell scripts calling themselves in cascade.
:spiral_note_pad: *Plan*:
*1/ (NOW) We merge our branch on `main` today, here is what it does
change for you 👇:skin-tone-3::*
* On `main`:
* It will introduce a new job on `main` _Bazel Push_, which will push
those new images on our registries with all tags prefixed by `bazel-`.
* These new images will be picked up by S2 and S2 only.
* The existing jobs building docker images and pushing them will stay in
place until we have QA'ed them enough and are confident to roll them out
on Dotcom.
* Because we'll be building both images, there will be more jobs running
on `main`, but this should not affect the wall clock time.
* On all branches (so your PRs and `main`)
* The _Bazel Test_ job will now run: Backend Integration Tests, E2E
Tests and CodeIntel QA
* This will increase the duration of your test jobs in PRs, but as we
haven't removed yet the `sg lint` step, it should not affect too much
the wall clock time of your PRs.
* But it will also increase your confidence toward your changes, as the
coverage will vastly increased compared to before.
* If you have ongoing branches which are affecting the docker images
(like adding a new binary, like the recent `scip-tags`, reach us out on
#job-fair-bazel so we can help you to port your changes. It's much much
simpler than before, but it's going to be unfamiliar to you).
* If something goes awfully wrong, we'll rollback and update this
thread.
*2/ (EOW / Early next week) Once we're confident enough with what we saw
on S2, we'll roll the new images on Dotcom.*
* After the first successful deploy and a few sanity checks, we will
drop the old images building jobs.
* At this point, we'll reach out to all TLs asking for their help to
exercise all features of our product to ensure we catch any potential
breakage.
## Test plan
<!-- All pull requests REQUIRE a test plan:
https://docs.sourcegraph.com/dev/background-information/testing_principles
-->
* We tested our new images on `scale-testing` and it worked.
* The new container building rules comes with _container tests_ which
ensures that produced images are containing and configured with what
should be in there:
[example](https://sourcegraph.sourcegraph.com/github.com/sourcegraph/sourcegraph@bzl/oci_wolfi/-/blob/enterprise/cmd/gitserver/image_test.yaml)
.
---------
Co-authored-by: Dave Try <davetry@gmail.com>
Co-authored-by: Will Dollman <will.dollman@sourcegraph.com>
