* initial change to use aspect-default and remove ifs
* use rosetta bazelrc in bazel ci scripts
* use /tmp/aspect-generated.bazelrc path everywhere
change gcp project depending on queue
* restore aspect buildkite plugin
* add pipeline conditional to buildkite.yaml
* split pipeline up into two groups depending on pipeline slug
* update commit we should be after
* update pipeline for upload
* update commit step key
* remove nested group
* change workflow upload step key
* fix command
* fix group name
Another step towards https://github.com/sourcegraph/sourcegraph/issues/59155, previously `bazel test //...` would error at analysis time on `//client/web/src/end-to-end:e2e` due to it attempting to perform variable substitution for env vars e.g. `"HEADLESS": "$(E2E_HEADLESS)"`, for values not defined via `--define` (we only set these explicitly in .aspect/bazelrc/ci.sourcegraph.bazelrc and some `sg` targets).
By leveraging https://bazel.build/rules/lib/builtins/actions#run.use_default_shell_env, we can allow the test to read values from `--action_env` while _also_ having explicit values set via `env` macro parameter. Previously, setting `env` macro parameter would completely shadow any `--action_env` values.
Unfortunately, we cant use `--test_env` for this, as `js_run_binary` is an action not a test (or something like that?).
We also cant do env renaming anymore, meaning we have to drop the `E2E_` prefix for some env vars. At least one script needed some reworking to accommodate that `e2e_test.sh`
## Test plan
👁️ CI once again 👁️
Cody no longer needs it and it is obsolete now!
Since App added a non-insignificant amount of new concepts and alternative code paths, I decided to take some time and remove it from out codebase.
This PR removes ~21k lines of code. If we ever want parts of single binary (app), the redis kv alternatives, or the release pipeline for a native mac app back, we can look back at this PR and revert parts of it, but maintaining 21k lines of code and many code paths for which I had to delete a surprisingly small amount of tests justifies this move for me very well.
Technically, to some extent SG App and Cody App both still existed in the codebase, but we don't distribute either of them anymore, so IMO we shouldn't keep this weight in our code.
So.. here we go.
This should not affect any of the existing deployments, we only remove functionality that was special-cased for app.
Our tracking of frontend (TypeScript) code coverage has decayed and was no longer being used. The Codecov report at https://app.codecov.io/gh/sourcegraph/sourcegraph significantly under-counts coverage, and what coverage tasks we did have were flaky in CI and relied on old libraries that have not been updated recently to keep with modern practices (such as `@storybook/addon-storyshots` only supporting Jest with `injectGlobals`).
From https://sourcegraph.slack.com/archives/C04MYFW01NV/p1698059032863009?thread_ts=1698054092.709309&cid=C04MYFW01NV, the proposal is to remove coverage tracking for now and prioritize getting tests to run really fast and smooth locally first, including upgrading our testing infra, and then reenable code coverage tracking.
We were not using [Lighthouse](https://developer.chrome.com/docs/lighthouse/overview/) reports, but they were running (asynchronously) on each build, which caused occasional build flakes and added a moderate amount of complexity to our CI.
The reports do have value in theory, but we were not actually consulting the reports.
This is the end of the PR train to remove the enterprise directory from out repo since we have consolidated to use a single license.
Bye rough code split :)
Fixes the error
```
/buildkite/builds/buildkite-agent-bazel-85dff99848-7lgqm-1/sourcegraph/sourcegraph/.buildkite/hooks/pre-exit: line 19: 0: command not found
--
| /buildkite/builds/buildkite-agent-bazel-85dff99848-7lgqm-1/sourcegraph/sourcegraph/.buildkite/hooks/pre-exit: line 22: -gt: command not found
```
## Test plan
Check the pre-exit output on any bazel agent in this PR
Only runs the docker clean up tasks if containers or volumes are found
to stop the error messages at the end of jobs
## Test plan
Works on this feature branch in CI
---------
Co-authored-by: William Bezuidenhout <william.bezuidenhout@sourcegraph.com>
We normally skip install of asdf images when GROUP_KEY is `Publish
Images`, except for certains steps:
* Publish executor image in
bc0049c818
and now also Publish Docker registry image
## Test plan
BUILDKITE_GROUP_KEY="Publishimages"
BUILDKITE_STEP_KEY="packerwhitecheckmarkPublishdockerregistrymirrorimage"
.buildkite/hooks/pre-command
<!-- All pull requests REQUIRE a test plan:
https://docs.sourcegraph.com/dev/background-information/testing_principles
-->
Use the pre-exit hook to clean up any running containers, as well as the
disk space if it's over `85%` to ensure the following run doesn't fail
## Test plan
New commands for disk tested manually on ubuntu machine
This reverts commit a52296cc49 - which
broke the app release pipeline:
```
$ trap 'kill -- $' INT TERM QUIT; buildkite-agent pipeline upload .buildkite/pipeline.app.yml
--
| 2023-05-22 18:52:04 INFO Reading pipeline config from ".buildkite/pipeline.app.yml"
| 2023-05-22 18:52:04 INFO Updating BUILDKITE_COMMIT to "0d43a156081ffe2fca34c912a4a575c77440c66f"
| 2023-05-22 18:52:05 WARN POST https://agent.buildkite.com/v3/jobs/018844cc-f041-44a4-9ceb-6863943f1473/pipelines: 422 `agent` is not a valid property on the `command` step, perhaps you meant `agents`? (Attempt 1/60 Retrying in 5s)
| 2023-05-22 18:52:05 ERROR Unrecoverable error, skipping retries
| 2023-05-22 18:52:05 FATAL Failed to upload and process pipeline: POST https://agent.buildkite.com/v3/jobs/018844cc-f041-44a4-9ceb-6863943f1473/pipelines: 422 `agent` is not a valid property on the `command` step, perhaps you meant `agents`?
| 🚨 Error: The command exited with status 1
| user command error: exit status 1
|
<br class="Apple-interchange-newline">
```
## Test plan
It worked before, so reverting.
This is mostly constrained to the `enterprise/dev/app` directory.
- [x] undo the bazel changes
- Builds the linux version on Bazel agents
- Builds arm64 version on macos
- Builds x86_64 version on macos using go - There is a fix to the bazel
scripts I have to make for this
- Bundles the linux versions on linux using Tauri
- Bundles the macos version on MacOS + Signs all the artefacts with
Apple
- Creates a GitHub release
## Future work
Currently a x86 on MacOS isn't published / bundled since we have to
install specific rust libraries for that
## Test plan
https://buildkite.com/sourcegraph/sourcegraph-app-release/builds?branch=wb%2Fapp%2Faws-macos
<!-- All pull requests REQUIRE a test plan:
https://docs.sourcegraph.com/dev/background-information/testing_principles
-->
`aws` is missing on `Publish Exeuctor Image` steps because the `asdf`
install step is skipped
## Test plan
```
sourcegraph on main [$!?] via 🐹 v1.19.8 took 5s
❯ BUILDKITE_GROUP_KEY="Publishimages" BUILDKITE_STEP_KEY="packerwhitecheckmarkPublishexecutoriage" .buildkite/hooks/pre-command
~~~ Preparing asdf dependencies
publish image step: skipping asdf install
sourcegraph on main [$!?] via 🐹 v1.19.8
❯ BUILDKITE_GROUP_KEY="Publishimages" BUILDKITE_STEP_KEY="packerwhitecheckmarkPublishexecutorimage" .buildkite/hooks/pre-command
~~~ Preparing asdf dependencies
running normal install
asdf install
fd plugin is not installed
shfmt plugin is not installed
shellcheck plugin is not installed
kubectl plugin is not installed
github-cli plugin is not installed
packer plugin is not installed
trivy plugin is not installed
kustomize plugin is not installed
awscli plugin is not installed
python plugin is not installed
rust plugin is not installed```
<!-- All pull requests REQUIRE a test plan: https://docs.sourcegraph.com/dev/background-information/testing_principles -->
Use volatile status variables provided by stamping to inject relevant
Git information into the Percy execution process. Percy needs Git commit
and branch to create an accurate report and auto-accept changes on main.
See extensive comments in the code for implementation details.
On the high level, instead of running the Percy CLI directly from Bazel,
we wrap it into the JS script that:
1. Reads volatile status variables with Git info.
2. Creates a Percy CLI command with these variables injected.
3. Exectes the command using args received from the Bazel target.
## Test plan
1. CI
2. Locally: `bazel test //client/web/src/integration:integration-tests
--define=E2E_HEADLESS=true
--define=E2E_SOURCEGRAPH_BASE_URL="http://localhost:7080"
--define=GH_TOKEN=fake --define=XXX` — replace `XXX` with the
`PERCY_TOKEN` from our shared 1Password vault.
Reintroduces the same changes as
https://github.com/sourcegraph/sourcegraph/pull/51104 minus
syntax-highlighter which we're unable to compile with the right
toolchain at the moment.
Tested as a full main-dry-run, as well as running the stack with compose
and checking indexing and syntax-highlighting.
Executors are also built correctly.
## Test plan
CI + manual test via compose.
---------
Co-authored-by: Jean-Hadrien Chabran <jh@chabran.fr>
Forgot to remove this previously.
Fixes these non build failing errors at the end of a build:
```
$ .buildkite/hooks/post-command
--
| couldn't parse startTime of ""
| couldn't parse startTime of ""
```
## Test plan
<!-- All pull requests REQUIRE a test plan:
https://docs.sourcegraph.com/dev/background-information/testing_principles
-->
CI
CI fails on main without this.
Test Plan: ran locally and modified gen-pipeline to confirm it prints
the correct cwd.
Co-authored-by: Jean-Hadrien Chabran <jh@chabran.fr>
- Run client E2E tests in Bazel
- Fix client integration tests execution locally
- Make client integration tests work with different
`SOURCEGRAPH_BASE_URL` values
- Clean up integration tests and E2E tests logs
## Test plan
1. CI
2. `bazel test //client/web/src/integration:integration-tests`
3. `bazel test //client/web/src/end-to-end:e2e`
---------
Co-authored-by: Jason Bedard <jason@aspect.dev>
Co-authored-by: William Bezuidenhout <william.bezuidenhout@sourcegraph.com>
Co-authored-by: Jean-Hadrien Chabran <jh@chabran.fr>
Co-authored-by: davejrt <davetry@gmail.com>
