artin/sourcegraph
1
0
Fork 0
mirror of https://github.com/sourcegraph/sourcegraph.git synced 2026-02-06 15:31:48 +00:00
Code Issues Packages Projects Releases Wiki Activity
58da6780d7
sourcegraph/WORKSPACE
Jean-Hadrien Chabran 58da6780d7
Switch to OCI/Wolfi based image (#52693) 
This PR ships our freshly rewritten container images built with
rules_oci and Wolfi, which for now will only be used on S2.

*What is this about*

This work is the conjunction of [hardening container
images](https://github.com/orgs/sourcegraph/projects/302?pane=issue&itemId=25019223)
and fully building our container images with Bazel.

* All base images are now distroless, based on Wolfi, meaning we fully
control every little package version and we won't be subject anymore to
Alpine maintainers dropping a postgres version for example.

* Container images are now built with `rules_oci`, meaning we don't have
Dockerfile anymore, but instead created through [Bazel
rules](https://sourcegraph.sourcegraph.com/github.com/sourcegraph/sourcegraph@bzl/oci_wolfi/-/blob/enterprise/cmd/gitserver/BUILD.bazel).
Don't be scared, while this will look a bit strange to you at first,
it's much saner and simpler to do than our Dockerfiles and their muddy
shell scripts calling themselves in cascade.


:spiral_note_pad:  *Plan*:

*1/ (NOW) We merge our branch on `main` today, here is what it does
change for you 👇:skin-tone-3::*

* On `main`: 
* It will introduce a new job on `main` _Bazel Push_, which will push
those new images on our registries with all tags prefixed by `bazel-`.
    * These new images will be picked up by S2 and S2 only. 
* The existing jobs building docker images and pushing them will stay in
place until we have QA'ed them enough and are confident to roll them out
on Dotcom.
* Because we'll be building both images, there will be more jobs running
on `main`, but this should not affect the wall clock time.
* On all branches (so your PRs and `main`)
* The _Bazel Test_ job will now run: Backend Integration Tests, E2E
Tests and CodeIntel QA
* This will increase the duration of your test jobs in PRs, but as we
haven't removed yet the `sg lint` step, it should not affect too much
the wall clock time of your PRs.
* But it will also increase your confidence toward your changes, as the
coverage will vastly increased compared to before.
* If you have ongoing branches which are affecting the docker images
(like adding a new binary, like the recent `scip-tags`, reach us out on
#job-fair-bazel so we can help you to port your changes. It's much much
simpler than before, but it's going to be unfamiliar to you).

* If something goes awfully wrong, we'll rollback and update this
thread.

*2/ (EOW / Early next week) Once we're confident enough with what we saw
on S2, we'll roll the new images on Dotcom.*

* After the first successful deploy and a few sanity checks, we will
drop the old images building jobs.
* At this point, we'll reach out to all TLs asking for their help to
exercise all features of our product to ensure we catch any potential
breakage.



## Test plan

<!-- All pull requests REQUIRE a test plan:
https://docs.sourcegraph.com/dev/background-information/testing_principles
-->


* We tested our new images on `scale-testing` and it worked.
* The new container building rules comes with _container tests_ which
ensures that produced images are containing and configured with what
should be in there:
[example](https://sourcegraph.sourcegraph.com/github.com/sourcegraph/sourcegraph@bzl/oci_wolfi/-/blob/enterprise/cmd/gitserver/image_test.yaml)
.

---------

Co-authored-by: Dave Try <davetry@gmail.com>
Co-authored-by: Will Dollman <will.dollman@sourcegraph.com>
2023-06-02 12:12:52 +02:00

382 lines
13 KiB
Python
Raw Blame History

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
name = "bazel_skylib",
sha256 = "74d544d96f4a5bb630d465ca8bbcfe231e3594e5aae57e1edbf17a6eb3ca2506",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.3.0/bazel-skylib-1.3.0.tar.gz",
"https://github.com/bazelbuild/bazel-skylib/releases/download/1.3.0/bazel-skylib-1.3.0.tar.gz",
],
)
load("@bazel_skylib//:workspace.bzl", "bazel_skylib_workspace")
bazel_skylib_workspace()
http_archive(
name = "aspect_bazel_lib",
sha256 = "0da75299c5a52737b2ac39458398b3f256e41a1a6748e5457ceb3a6225269485",
strip_prefix = "bazel-lib-1.31.2",
url = "https://github.com/aspect-build/bazel-lib/releases/download/v1.31.2/bazel-lib-v1.31.2.tar.gz",
)
http_archive(
name = "aspect_rules_js",
sha256 = "0b69e0967f8eb61de60801d6c8654843076bf7ef7512894a692a47f86e84a5c2",
strip_prefix = "rules_js-1.27.1",
url = "https://github.com/aspect-build/rules_js/releases/download/v1.27.1/rules_js-v1.27.1.tar.gz",
)
http_archive(
name = "aspect_rules_ts",
sha256 = "ace5b609603d9b5b875d56c9c07182357c4ee495030f40dcefb10d443ba8c208",
strip_prefix = "rules_ts-1.4.0",
url = "https://github.com/aspect-build/rules_ts/releases/download/v1.4.0/rules_ts-v1.4.0.tar.gz",
)
http_archive(
name = "aspect_rules_jest",
sha256 = "bf8f4a4d2a833e4f96f866c686c38bcee69d3bdae8a827b1c9d2fdf92212bc0b",
strip_prefix = "rules_jest-95d8f1961a9c6f3aee2929881b1b74461652e775",
url = "https://github.com/aspect-build/rules_jest/archive/95d8f1961a9c6f3aee2929881b1b74461652e775.tar.gz",
)
http_archive(
name = "io_bazel_rules_go",
sha256 = "dd926a88a564a9246713a9c00b35315f54cbd46b31a26d5d8fb264c07045f05d",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.38.1/rules_go-v0.38.1.zip",
"https://github.com/bazelbuild/rules_go/releases/download/v0.38.1/rules_go-v0.38.1.zip",
],
)
http_archive(
name = "rules_buf",
sha256 = "523a4e06f0746661e092d083757263a249fedca535bd6dd819a8c50de074731a",
strip_prefix = "rules_buf-0.1.1",
urls = [
"https://github.com/bufbuild/rules_buf/archive/refs/tags/v0.1.1.zip",
],
)
http_archive(
name = "bazel_gazelle",
sha256 = "ecba0f04f96b4960a5b250c8e8eeec42281035970aa8852dda73098274d14a1d",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/bazel-gazelle/releases/download/v0.29.0/bazel-gazelle-v0.29.0.tar.gz",
"https://github.com/bazelbuild/bazel-gazelle/releases/download/v0.29.0/bazel-gazelle-v0.29.0.tar.gz",
],
)
http_archive(
name = "rules_rust",
sha256 = "dc8d79fe9a5beb79d93e482eb807266a0e066e97a7b8c48d43ecf91f32a3a8f3",
urls = ["https://github.com/bazelbuild/rules_rust/releases/download/0.19.0/rules_rust-v0.19.0.tar.gz"],
)
# Container rules
http_archive(
name = "rules_oci",
sha256 = "db57efd706f01eb3ce771468366baa1614b5b25f4cce99757e2b8d942155b8ec",
strip_prefix = "rules_oci-1.0.0",
url = "https://github.com/bazel-contrib/rules_oci/releases/download/v1.0.0/rules_oci-v1.0.0.tar.gz",
)
http_archive(
name = "rules_pkg",
sha256 = "8c20f74bca25d2d442b327ae26768c02cf3c99e93fad0381f32be9aab1967675",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/rules_pkg/releases/download/0.8.1/rules_pkg-0.8.1.tar.gz",
"https://github.com/bazelbuild/rules_pkg/releases/download/0.8.1/rules_pkg-0.8.1.tar.gz",
],
)
SRC_CLI_VERSION = "5.0.3"
http_archive(
name = "src-cli-linux-amd64",
sha256 = "d125d732ad4c47ae6977c49574b01cc1b3c943b2a2108142267438e829538aa3",
url = "https://github.com/sourcegraph/src-cli/releases/download/{0}/src-cli_{0}_linux_amd64.tar.gz".format(SRC_CLI_VERSION),
build_file_content = """
filegroup(
name = "src-cli-linux-amd64",
srcs = ["src"],
visibility = ["//visibility:public"],
)
"""
)
http_archive(
name = "container_structure_test",
sha256 = "42edb647b51710cb917b5850380cc18a6c925ad195986f16e3b716887267a2d7",
strip_prefix = "container-structure-test-104a53ede5f78fff72172639781ac52df9f5b18f",
urls = ["https://github.com/GoogleContainerTools/container-structure-test/archive/104a53ede5f78fff72172639781ac52df9f5b18f.zip"],
)
# hermetic_cc_toolchain setup ================================
HERMETIC_CC_TOOLCHAIN_VERSION = "v2.0.0-rc2"
http_archive(
name = "hermetic_cc_toolchain",
sha256 = "40dff82816735e631e8bd51ede3af1c4ed1ad4646928ffb6a0e53e228e55738c",
urls = [
"https://mirror.bazel.build/github.com/uber/hermetic_cc_toolchain/releases/download/{0}/hermetic_cc_toolchain-{0}.tar.gz".format(HERMETIC_CC_TOOLCHAIN_VERSION),
"https://github.com/uber/hermetic_cc_toolchain/releases/download/{0}/hermetic_cc_toolchain-{0}.tar.gz".format(HERMETIC_CC_TOOLCHAIN_VERSION),
],
)
# rules_js setup ================================
load("@aspect_rules_js//js:repositories.bzl", "rules_js_dependencies")
rules_js_dependencies()
# node toolchain setup ==========================
load("@rules_nodejs//nodejs:repositories.bzl", "nodejs_register_toolchains")
nodejs_register_toolchains(
name = "nodejs",
node_version = "16.19.0",
)
# rules_js npm setup ============================
load("@aspect_rules_js//npm:npm_import.bzl", "npm_translate_lock")
npm_translate_lock(
name = "npm",
npm_package_target_name = "{dirname}_pkg",
npmrc = "//:.npmrc",
pnpm_lock = "//:pnpm-lock.yaml",
# Required for ESLint test targets.
# See https://github.com/aspect-build/rules_js/issues/239
# See `public-hoist-pattern[]=*eslint*` in the `.npmrc` of this monorepo.
public_hoist_packages = {
"@typescript-eslint/eslint-plugin": [""],
"@typescript-eslint/parser@5.56.0_qxbo2xm47qt6fxnlmgbosp4hva": [""],
"eslint-config-prettier": [""],
"eslint-plugin-ban": [""],
"eslint-plugin-etc": [""],
"eslint-plugin-import": [""],
"eslint-plugin-jest-dom": [""],
"eslint-plugin-jsdoc": [""],
"eslint-plugin-jsx-a11y": [""],
"eslint-plugin-react@7.32.1_eslint_8.34.0": [""],
"eslint-plugin-react-hooks": [""],
"eslint-plugin-rxjs": [""],
"eslint-plugin-unicorn": [""],
"eslint-plugin-unused-imports": [""],
"eslint-import-resolver-node": [""],
},
verify_node_modules_ignored = "//:.bazelignore",
)
# rules_ts npm setup ============================
load("@npm//:repositories.bzl", "npm_repositories")
npm_repositories()
load("@aspect_rules_ts//ts:repositories.bzl", "rules_ts_dependencies")
rules_ts_dependencies(ts_version = "4.9.5")
# rules_jest setup ==============================
load("@aspect_rules_jest//jest:dependencies.bzl", "rules_jest_dependencies")
rules_jest_dependencies()
load("@aspect_rules_jest//jest:repositories.bzl", "jest_repositories")
jest_repositories(
name = "jest",
jest_version = "v28.1.0",
)
load("@jest//:npm_repositories.bzl", jest_npm_repositories = "npm_repositories")
jest_npm_repositories()
# rules_esbuild setup ===========================
http_archive(
name = "aspect_rules_esbuild",
sha256 = "2ea31bd97181a315e048be693ddc2815fddda0f3a12ca7b7cc6e91e80f31bac7",
strip_prefix = "rules_esbuild-0.14.4",
url = "https://github.com/aspect-build/rules_esbuild/releases/download/v0.14.4/rules_esbuild-v0.14.4.tar.gz",
)
load("@aspect_rules_esbuild//esbuild:dependencies.bzl", "rules_esbuild_dependencies")
rules_esbuild_dependencies()
# Register a toolchain containing esbuild npm package and native bindings
load("@aspect_rules_esbuild//esbuild:repositories.bzl", "LATEST_VERSION", "esbuild_register_toolchains")
esbuild_register_toolchains(
name = "esbuild",
esbuild_version = LATEST_VERSION,
)
# rules_webpack setup ===========================
# Commit to include unreleased https://github.com/aspect-build/rules_webpack/commit/4a5f04a4bc504f71d32825124c7872ff721aa1b0
http_archive(
name = "aspect_rules_webpack",
sha256 = "8d81f8d018127c72270ea4b7287be5c4ff63d9656a34334c305d52f14e0c922f",
strip_prefix = "rules_webpack-4a5f04a4bc504f71d32825124c7872ff721aa1b0",
url = "https://github.com/aspect-build/rules_webpack/archive/4a5f04a4bc504f71d32825124c7872ff721aa1b0.tar.gz",
)
load("@aspect_rules_webpack//webpack:dependencies.bzl", "rules_webpack_dependencies")
rules_webpack_dependencies()
load("@aspect_rules_webpack//webpack:repositories.bzl", "webpack_repositories")
webpack_repositories(name = "webpack")
load("@webpack//:npm_repositories.bzl", webpack_npm_repositories = "npm_repositories")
webpack_npm_repositories()
# Go toolchain setup
load("@rules_buf//buf:repositories.bzl", "rules_buf_dependencies", "rules_buf_toolchains")
rules_buf_dependencies()
rules_buf_toolchains(version = "v1.11.0")
load("@rules_buf//gazelle/buf:repositories.bzl", "gazelle_buf_dependencies")
gazelle_buf_dependencies()
load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies", "rules_proto_toolchains")
rules_proto_dependencies()
rules_proto_toolchains()
load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies")
load("@bazel_gazelle//:deps.bzl", "gazelle_dependencies", "go_repository")
load("//:linter_deps.bzl", "linter_dependencies")
load("//:deps.bzl", "go_dependencies")
go_repository(
name = "com_github_aws_aws_sdk_go_v2_service_ssooidc",
build_file_proto_mode = "disable_global",
importpath = "github.com/aws/aws-sdk-go-v2/service/ssooidc",
sum = "h1:0bLhH6DRAqox+g0LatcjGKjjhU6Eudyys6HB6DJVPj8=",
version = "v1.14.1",
)
# Overrides the default provided protobuf dep from rules_go by a more
# recent one.
go_repository(
name = "org_golang_google_protobuf",
build_file_proto_mode = "disable_global",
importpath = "google.golang.org/protobuf",
sum = "h1:7QBf+IK2gx70Ap/hDsOmam3GE0v9HicjfEdAxE62UoM=",
version = "v1.29.1",
) # keep
# Pin protoc-gen-go-grpc to 1.3.0
# See also //:gen-go-grpc
go_repository(
name = "org_golang_google_grpc_cmd_protoc_gen_go_grpc",
build_file_proto_mode = "disable_global",
importpath = "google.golang.org/grpc/cmd/protoc-gen-go-grpc",
sum = "h1:rNBFJjBCOgVr9pWD7rs/knKL4FRTKgpZmsRfV214zcA=",
version = "v1.3.0",
) # keep
# gazelle:repository_macro deps.bzl%go_dependencies
go_dependencies()
go_rules_dependencies()
go_register_toolchains(
nogo = "@//:sg_nogo",
version = "1.19.8",
)
linter_dependencies()
gazelle_dependencies()
load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")
protobuf_deps()
# rust toolchain setup
load("@rules_rust//rust:repositories.bzl", "rules_rust_dependencies", "rust_register_toolchains")
rules_rust_dependencies()
rust_register_toolchains(
edition = "2021",
# Keep in sync with docker-images/syntax-highlighter/Dockerfile
# and docker-images/syntax-highlighter/rust-toolchain.toml
versions = [
"1.68.0",
],
)
load("@rules_rust//crate_universe:defs.bzl", "crates_repository")
crates_repository(
name = "crate_index",
cargo_config = "//docker-images/syntax-highlighter:.cargo/config.toml",
cargo_lockfile = "//docker-images/syntax-highlighter:Cargo.lock",
# this file has to be manually created and it will be filled when
# the target is ran.
# To regenerate this file run: CARGO_BAZEL_REPIN=1 bazel sync --only=crate_index
lockfile = "//docker-images/syntax-highlighter:Cargo.Bazel.lock",
# glob doesn't work in WORKSPACE files: https://github.com/bazelbuild/bazel/issues/11935
manifests = [
"//docker-images/syntax-highlighter:Cargo.toml",
"//docker-images/syntax-highlighter:crates/scip-macros/Cargo.toml",
"//docker-images/syntax-highlighter:crates/scip-syntax/Cargo.toml",
"//docker-images/syntax-highlighter:crates/scip-treesitter/Cargo.toml",
"//docker-images/syntax-highlighter:crates/scip-treesitter-languages/Cargo.toml",
"//docker-images/syntax-highlighter:crates/sg-syntax/Cargo.toml",
],
)
load("@crate_index//:defs.bzl", "crate_repositories")
crate_repositories()
load("@hermetic_cc_toolchain//toolchain:defs.bzl", zig_toolchains = "toolchains")
zig_toolchains()
load("//dev/backcompat:defs.bzl", "back_compat_defs")
back_compat_defs()
# containers steup ===============================
load("@rules_oci//oci:dependencies.bzl", "rules_oci_dependencies")
rules_oci_dependencies()
load("@rules_oci//oci:repositories.bzl", "LATEST_CRANE_VERSION", "LATEST_ZOT_VERSION", "oci_register_toolchains")
oci_register_toolchains(
name = "oci",
crane_version = LATEST_CRANE_VERSION,
# Uncommenting the zot toolchain will cause it to be used instead of crane for some tasks.
# Note that it does not support docker-format images.
# zot_version = LATEST_ZOT_VERSION,
)
# Optional, for oci_tarball rule
load("@rules_pkg//:deps.bzl", "rules_pkg_dependencies")
rules_pkg_dependencies()
load("//dev:oci_deps.bzl", "oci_deps")
oci_deps()
load("//enterprise/cmd/embeddings/shared:assets.bzl", "embbedings_assets_deps")
embbedings_assets_deps()
load("@container_structure_test//:repositories.bzl", "container_structure_test_register_toolchain")
container_structure_test_register_toolchain(name = "cst")
Reference in New Issue View Git Blame Copy Permalink