* Remove qdrant package for testing
* Soft-fail if no apkfragments were found
* Bump docker-client for testing
* Revert "Remove qdrant package for testing"
This reverts commit 4c9eac7399.
* Revert "Bump docker-client for testing"
This reverts commit 39fad85086.
* wip
* gitserver (mostly) wolfi 4 bazel
* the big heck of all things
* Add rules_apko lock translation rules to WORKSPACE
* Call apko_repositories() more
* fix rules_apko to handle our shorter repo urls
* fix workspace from rebase, and missing locks
* visibility on wolfi_base_image
* hand-fix a lock coz apko lock is 🅱️roken
* remove chainguard repo+keyring from base
* update locks
* add chainguard repo+keychain to single server manifest
* unrelated fixes, server+grafana still h*cked
* fix postgres-exporter
* the big fix
* aws lib got bumped?
* downgrade sso-oidc? idk
* ignore wolfi locks from prettier
* dynamically do the locks with a reporule
* document and make nice :nails:
* bazel run @rules_apko//apko patch
* Fix .typo.typo
* Update tooling for end-to-end Bazel images (#61106)
* Update sg wolfi image to build using Bazel
* bazel run @rules_apko//apko patch
* Fix .typo.typo
* Add update-images and implement apko YAML change monitoring
* Use bazel apko and add support for additional repos
* Refactor sg wolfi
* Rework wolfi base image auto-update pipeline
* sg bazel configure
* [rough] Add --check flag to sg wolfi lock
* Refactor sg wolfi lock --check
* Simplify check and update apko lock hash operations
* Fix resolveImagePath when running in bazel
* Fixup logic error in CheckApkoLockHashes
* Tweak DoBaseImageBuild output
* Remove debug output
* Fix sg wolfi lock --check behaviour for all images
* Replace base image build step with apko lock --check
* Remove debug line
* Minor fixups for CI step
* Wrap with AnnotatedCmd
* Fixup annotation
* Update apko lockfiles
* Allow additional repos to be passed
* Update build-base-image.sh with bazel + add back to pipeline
* Ensure that modified base images are rebuilt
* Solve bazelception
* Remove timestamp for bit-level reproducibility
* Skip local keygen when running on buildkite
* Add workaround for lack of local repo support in rules_apko
* Run apkoOps first as it's quick and might fail
* Remove blocking allBaseImagesBuilt step
* Remove unused promethus-gcp image
* Add special cases to resolveImagePath
* Cleanly handle case where no bazel build path exists
This could happen in cases where a base image is only used outside of sourcegraph/sourcegraph,
or if you've added a new base image config but haven't added the associated Bazel scaffolding
* Add debugging around failing docker builds
* More debugging
* Normalise apko_lockfile to match repo.bzl
* Fixup apko docker call
* Try passing imageconfigdir differently to docker
* Run ls in different container
* Soft-fail when using legacy build in Buildkite
* Add missing include
* Workaround for building sourcegraph and sourcegraph-dev
* Add postgresql-client package to server
This contains createdb, which was recently moved from postgresql
* Inflate postgres-12-codeinsights image to avoid rules_apko errors
* Remove update line from yaml files
* Fix issue caused by moving base sourcegraph image
* Remove apk-tools from server
* Update lockfiles
* Address review feedback
* Remove debug lines
* fix unbound var
---------
Co-authored-by: Noah Santschi-Cooney <noah@santschi-cooney.ch>
* go mod tidy + gazelle-update-repos after merging main
* Use aspect bazel cache
* Use Aspect bazel caching when calling bazel in bash and sg
* Append annotation
* Run apko lock on aspect agent
* Remove base image builds
Discussion in https://sourcegraph.slack.com/archives/C05EVRLQEUR/p1712307465660509
* Remove unused functionality
* Update BaseImageConfig comments
* Rewrite wolfi-images/README.md
* Add .apko/range.sh to .gitattributes
* Remove "wolfi" from :base_image and :base_tarball targets
* remove allowlist extras from debugging
* Tweak user instructions around package testing
* Add agent healthcheck to buildkite scripts
* prettier
* sg bazel configure
* bazel run //:gazelle-update-repos
---------
Co-authored-by: Noah Santschi-Cooney <noah@santschi-cooney.ch>
Co-authored-by: Noah S-C <noah@sourcegraph.com>
* wip
* gitserver (mostly) wolfi 4 bazel
* the big heck of all things
* Add rules_apko lock translation rules to WORKSPACE
* Call apko_repositories() more
* fix rules_apko to handle our shorter repo urls
* fix workspace from rebase, and missing locks
* visibility on wolfi_base_image
* hand-fix a lock coz apko lock is 🅱️roken
* remove chainguard repo+keyring from base
* update locks
* add chainguard repo+keychain to single server manifest
* unrelated fixes, server+grafana still h*cked
* fix postgres-exporter
* the big fix
* aws lib got bumped?
* downgrade sso-oidc? idk
* ignore wolfi locks from prettier
* dynamically do the locks with a reporule
* document and make nice :nails:
* bazel run @rules_apko//apko patch
* Fix .typo.typo
* Update tooling for end-to-end Bazel images (#61106)
* Update sg wolfi image to build using Bazel
* bazel run @rules_apko//apko patch
* Fix .typo.typo
* Add update-images and implement apko YAML change monitoring
* Use bazel apko and add support for additional repos
* Refactor sg wolfi
* Rework wolfi base image auto-update pipeline
* sg bazel configure
* [rough] Add --check flag to sg wolfi lock
* Refactor sg wolfi lock --check
* Simplify check and update apko lock hash operations
* Fix resolveImagePath when running in bazel
* Fixup logic error in CheckApkoLockHashes
* Tweak DoBaseImageBuild output
* Remove debug output
* Fix sg wolfi lock --check behaviour for all images
* Replace base image build step with apko lock --check
* Remove debug line
* Minor fixups for CI step
* Wrap with AnnotatedCmd
* Fixup annotation
* Update apko lockfiles
* Allow additional repos to be passed
* Update build-base-image.sh with bazel + add back to pipeline
* Ensure that modified base images are rebuilt
* Solve bazelception
* Remove timestamp for bit-level reproducibility
* Skip local keygen when running on buildkite
* Add workaround for lack of local repo support in rules_apko
* Run apkoOps first as it's quick and might fail
* Remove blocking allBaseImagesBuilt step
* Remove unused promethus-gcp image
* Add special cases to resolveImagePath
* Cleanly handle case where no bazel build path exists
This could happen in cases where a base image is only used outside of sourcegraph/sourcegraph,
or if you've added a new base image config but haven't added the associated Bazel scaffolding
* Add debugging around failing docker builds
* More debugging
* Normalise apko_lockfile to match repo.bzl
* Fixup apko docker call
* Try passing imageconfigdir differently to docker
* Run ls in different container
* Soft-fail when using legacy build in Buildkite
* Add missing include
* Workaround for building sourcegraph and sourcegraph-dev
* Add postgresql-client package to server
This contains createdb, which was recently moved from postgresql
* Inflate postgres-12-codeinsights image to avoid rules_apko errors
* Remove update line from yaml files
* Fix issue caused by moving base sourcegraph image
* Remove apk-tools from server
* Update lockfiles
* Address review feedback
* Remove debug lines
* fix unbound var
---------
Co-authored-by: Noah Santschi-Cooney <noah@santschi-cooney.ch>
* go mod tidy + gazelle-update-repos after merging main
* Use aspect bazel cache
* Use Aspect bazel caching when calling bazel in bash and sg
* Append annotation
* Run apko lock on aspect agent
* Remove base image builds
Discussion in https://sourcegraph.slack.com/archives/C05EVRLQEUR/p1712307465660509
* Remove unused functionality
* Update BaseImageConfig comments
* Rewrite wolfi-images/README.md
* Add .apko/range.sh to .gitattributes
* Remove "wolfi" from :base_image and :base_tarball targets
* remove allowlist extras from debugging
* Tweak user instructions around package testing
* Add agent healthcheck to buildkite scripts
* prettier
---------
Co-authored-by: Noah Santschi-Cooney <noah@santschi-cooney.ch>
Co-authored-by: Noah S-C <noah@sourcegraph.com>
See https://github.com/orgs/Homebrew/discussions/4686#discussioncomment-6628463
Thanks @jac !
## Test plan
- Ensure CI builds correctly
<!-- All pull requests REQUIRE a test plan: https://docs.sourcegraph.com/dev/background-information/testing_principles
Why does it matter?
These test plans are there to demonstrate that are following industry standards which are important or critical for our customers.
They might be read by customers or an auditor. There are meant be simple and easy to read. Simply explain what you did to ensure
your changes are correct!
Here are a non exhaustive list of test plan examples to help you:
- Making changes on a given feature or component:
- "Covered by existing tests" or "CI" for the shortest possible plan if there is zero ambiguity
- "Added new tests"
- "Manually tested" (if non trivial, share some output, logs, or screenshot)
- Updating docs:
- "previewed locally"
- share a screenshot if you want to be thorough
- Updating deps, that would typically fail immediately in CI if incorrect
- "CI"
- "locally tested"
-->
* initial change to use aspect-default and remove ifs
* use rosetta bazelrc in bazel ci scripts
* use /tmp/aspect-generated.bazelrc path everywhere
change gcp project depending on queue
* restore aspect buildkite plugin
* Bump all packages to sign them
* Update hashes and other minor issues
* Remove unused syntect-server package
* Revert "Remove unused syntect-server package"
This reverts commit 073e7b38da.
* Undo syntect-server epoch bump
Differences in flags are causing a lot of analysis cache discarding in CI due to differences.
We mightnt see the results for this until every branch has these changes, as bazel only keeps one analysis cache around.
But also sometimes builds are very fast even when analysis cache is busted 🙃 so who knows
## Test plan
👁️ Observing sourcegraph & aspect pipelines 👁️
Closes https://github.com/sourcegraph/sourcegraph/issues/54829
Next steps:
- what safeguards will we have in place for when we remove this from `sg lint`
- should we remove it from `sg lint`, its likely not contributing much overhead even in the uncommon case of shell files being touched
_How did some of these violations get in when theres shell lints :clueless:_
## Test plan
`pre-commit run --hook-stage=pre-push shellcheck shfmt`
Cody no longer needs it and it is obsolete now!
Since App added a non-insignificant amount of new concepts and alternative code paths, I decided to take some time and remove it from out codebase.
This PR removes ~21k lines of code. If we ever want parts of single binary (app), the redis kv alternatives, or the release pipeline for a native mac app back, we can look back at this PR and revert parts of it, but maintaining 21k lines of code and many code paths for which I had to delete a surprisingly small amount of tests justifies this move for me very well.
Technically, to some extent SG App and Cody App both still existed in the codebase, but we don't distribute either of them anymore, so IMO we shouldn't keep this weight in our code.
So.. here we go.
This should not affect any of the existing deployments, we only remove functionality that was special-cased for app.
* add wolfi annotation ctx
* fix formatting of annoation and use annotatedCmd
* add job output link
* Update wolfi-images/cadvisor.yaml
Co-authored-by: Will Dollman <will.dollman@sourcegraph.com>
---------
Co-authored-by: Will Dollman <will.dollman@sourcegraph.com>
* `gh` is now included in the base image by default
* Add release branch backport label
* Update image update docs
* Tweak PR metadata
* Print go version to debug weird error
* Comment out pipeline for faster debugging
* Remove pipeline dep
* Fix go run
* Temporarily re-add github cli fetch
* Revert pipeline optimisation
* Add pipeline step to run sg wolfi update-hashes
* Remove unused variable
* Testing sg version
* Update sg command
* Test gh cli client
* Ad-hoc install gh-cli
This will later be added to the base agents, so this is just a temporary step
* Commit changes to oci_deps and try using gh to fetch PRs
* Remove sg debug commands
* Fox typoo
* Delete branch if it already exists
This might cause problems with stateful runners - need to confirm
* Add debugging
* Tweak github PR search
* Catch potential error in git br -D
* Enable push and PR creation
* Tweak PR metadata
* Add test plan to PR
* Use a multi-line string
* Remove debug comments
* Comment out unused variable
* Replace `cat` with `git diff` to show changes
* Quiet grep
Co-authored-by: William Bezuidenhout <william.bezuidenhout@sourcegraph.com>
* Add git emoji in output
Co-authored-by: William Bezuidenhout <william.bezuidenhout@sourcegraph.com>
* Add github icon to output
Co-authored-by: William Bezuidenhout <william.bezuidenhout@sourcegraph.com>
---------
Co-authored-by: William Bezuidenhout <william.bezuidenhout@sourcegraph.com>
* Bump base images
* wolfi: print cwd on failure
* wolfi: fix cwd
Because of the recent move we went up one directory too much, leading to
the Wolfi scripts failing.
---------
Co-authored-by: Vincent Ruijter <vincent.ruijter@sourcegraph.com>
Co-authored-by: Vincent <evict@users.noreply.github.com>
This is the end of the PR train to remove the enterprise directory from out repo since we have consolidated to use a single license.
Bye rough code split :)
