sourcegraph/wolfi-images
Craig Furman d24e8fe7f3
feat(appliance): backport all recent appliance changes (#64182)
Draft in case plan in
https://linear.app/sourcegraph/issue/REL-309/release-process-for-appliance
not agreed. Please see that first.

Generated by:

```
git log --format=%H d47b4cc48b6ea27cf6b5a274b79a6a4c8f38cf8c..origin/main -- cmd/appliance internal/appliance docker-images/appliance-frontend | tac | xargs git cherry-pick
```

d47b4cc48b being the commit we branched
off main from to create the 5.5.x branch
(https://buildkite.com/sourcegraph/sourcegraph/builds/281882).

Commits (generated by `git log --format='-
https://github.com/sourcegraph/sourcegraph/commit/%H'
d47b4cc48b6ea27cf6b5a274b79a6a4c8f38cf8c..origin/main -- cmd/appliance
internal/appliance docker-images/appliance-frontend | tac`):

-
a20b0650b4
-
b71c986c77
-
91864283bc
-
c88b57020f
-
0491839942
-
619fc57074
-
e81c39a834
-
a61f353e0e
-
0abef7b43d
-
0e391a964a
-
daae9adfb6
-
6e31f0f4cc
-
49a600220d
-
37cf4a7b7e
-
29fc613c37
-
255e6387cc
-
49b32fcf3a
-
9f4c160f91
-
3814fd7390
-
c68e92bc28
-
7e82c27ab5
-
98c6b9703f
-
a01ebad841
-
8c2d8da234
-
ebec72d7ed
-
d945f19285
-
84e28998e9


## Test plan

Tests pass.

## Changelog

- Backport all recent appliance changes. The appliance is still
pre-release.

---------

Co-authored-by: Jacob Pleiness <jdpleiness@users.noreply.github.com>
Co-authored-by: Anish Lakhwara <anish+github@lakhwara.com>
Co-authored-by: Warren Gifford <warren@sourcegraph.com>
Co-authored-by: Nelson Araujo <nelsonjr@users.noreply.github.com>
2024-07-31 17:26:56 +00:00
..
cloud-mi2-base Publish a cloud-mi2 base image for use in the cloud repo (#62046) 2024-04-26 11:59:53 +01:00
sourcegraph-base Publish sourcegraph-base image (#62002) 2024-04-19 14:44:06 +01:00
sourcegraph-dev Publish sourcegraph-dev image (#63031) 2024-06-03 14:41:54 +01:00
.gitignore Add sg commands to build wolfi packages and images locally (#55316) 2023-07-28 09:23:25 +01:00
appliance-frontend.lock.json feat(appliance): backport all recent appliance changes (#64182) 2024-07-31 17:26:56 +00:00
appliance-frontend.yaml feat(appliance): backport all recent appliance changes (#64182) 2024-07-31 17:26:56 +00:00
appliance.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
appliance.yaml feat(appliance): appliance docker container (#63357) 2024-06-27 17:09:13 +00:00
batcheshelper.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
batcheshelper.yaml Revert publishing wolfi base images (#62117) 2024-04-25 11:33:00 +01:00
blobstore.lock.json Bump openjdk-11 version in blobstore image (#64047) 2024-07-24 16:20:47 +00:00
blobstore.yaml Revert publishing wolfi base images (#62117) 2024-04-25 11:33:00 +01:00
BUILD.bazel Build images end-to-end using Bazel v2 (#61845) 2024-04-12 16:18:43 +01:00
bundled-executor.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
bundled-executor.yaml Revert publishing wolfi base images (#62117) 2024-04-25 11:33:00 +01:00
caddy.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
caddy.yaml Revert publishing wolfi base images (#62117) 2024-04-25 11:33:00 +01:00
cadvisor.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
cadvisor.yaml Revert publishing wolfi base images (#62117) 2024-04-25 11:33:00 +01:00
cloud-mi2.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
cloud-mi2.yaml Revert publishing wolfi base images (#62117) 2024-04-25 11:33:00 +01:00
defs.bzl chore(bazel): don't stamp yq rule for wolfi base images (#63470) 2024-06-25 16:19:45 +01:00
executor-kubernetes.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
executor-kubernetes.yaml Revert publishing wolfi base images (#62117) 2024-04-25 11:33:00 +01:00
executor.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
executor.yaml Revert publishing wolfi base images (#62117) 2024-04-25 11:33:00 +01:00
gitserver.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
gitserver.yaml Revert publishing wolfi base images (#62117) 2024-04-25 11:33:00 +01:00
grafana.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
grafana.yaml Revert publishing wolfi base images (#62117) 2024-04-25 11:33:00 +01:00
jaeger-agent.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
jaeger-agent.yaml Revert publishing wolfi base images (#62117) 2024-04-25 11:33:00 +01:00
jaeger-all-in-one.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
jaeger-all-in-one.yaml Revert publishing wolfi base images (#62117) 2024-04-25 11:33:00 +01:00
node-exporter.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
node-exporter.yaml Revert publishing wolfi base images (#62117) 2024-04-25 11:33:00 +01:00
opentelemetry-collector.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
opentelemetry-collector.yaml Revert publishing wolfi base images (#62117) 2024-04-25 11:33:00 +01:00
postgres-exporter.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
postgres-exporter.yaml Revert publishing wolfi base images (#62117) 2024-04-25 11:33:00 +01:00
postgresql-12-codeinsights.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
postgresql-12-codeinsights.yaml Revert publishing wolfi base images (#62117) 2024-04-25 11:33:00 +01:00
postgresql-12.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
postgresql-12.yaml Revert publishing wolfi base images (#62117) 2024-04-25 11:33:00 +01:00
prometheus.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
prometheus.yaml Revert publishing wolfi base images (#62117) 2024-04-25 11:33:00 +01:00
README.md fix: update links for dev docs (#62758) 2024-05-17 13:47:34 +02:00
rebuild-images.sh Wolfi - simplify images (#52138) 2023-05-19 15:00:14 -04:00
redis-exporter.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
redis-exporter.yaml Revert publishing wolfi base images (#62117) 2024-04-25 11:33:00 +01:00
redis.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
redis.yaml Revert publishing wolfi base images (#62117) 2024-04-25 11:33:00 +01:00
repo-updater.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
repo-updater.yaml Revert publishing wolfi base images (#62117) 2024-04-25 11:33:00 +01:00
repo.bzl Build our own caddy image (#61881) 2024-04-15 17:10:14 +01:00
search-indexer.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
search-indexer.yaml Revert publishing wolfi base images (#62117) 2024-04-25 11:33:00 +01:00
searcher.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
searcher.yaml Revert publishing wolfi base images (#62117) 2024-04-25 11:33:00 +01:00
server.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
server.yaml Revert publishing wolfi base images (#62117) 2024-04-25 11:33:00 +01:00
sourcegraph-base.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
sourcegraph-base.yaml Revert publishing wolfi base images (#62117) 2024-04-25 11:33:00 +01:00
sourcegraph-dev.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
sourcegraph-dev.yaml Publish sourcegraph-dev image (#63031) 2024-06-03 14:41:54 +01:00
sourcegraph-template.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
sourcegraph-template.yaml Upgrade git to 2.45.0 and unpin old cURL version (#62282) 2024-04-30 17:52:52 +02:00
symbols.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
symbols.yaml Revert publishing wolfi base images (#62117) 2024-04-25 11:33:00 +01:00
syntax-highlighter.lock.json security: Auto-update package lockfiles for Sourcegraph base images (#64035) 2024-07-24 11:46:40 +00:00
syntax-highlighter.yaml Revert publishing wolfi base images (#62117) 2024-04-25 11:33:00 +01:00

Wolfi base images for Sourcegraph containers

Rather than building our containers on top of an upstream image like alpine:latest, at Sourcegraph we build our own containers entirely from scratch using Bazel and apko.

This directory contains the configuration for each of our base images. Base images contain all the dependencies that the various components of Sourcegraph require in order to run, such as packages, users, groups, directores, and environment variables. For example, the gitserver configuration file ensures that Git is installed.

To create the final images that are shipped and deployed, we take the base image and use Bazel to build and add our own binaries on top.

The structure of this directory is:

  • <image>.yaml - apko configuration that declares the set of packages, users & groups, directories, and envars for each base image
  • <image>.lock.json - a lockfile which contains precise versions and hashes of packages, used by Bazel for reproducible builds. Generated from <image>.yaml using sg wolfi lock.

Getting started

See the Add and Update Wolfi Base Images docs for guides to add new images and updating existing images. For more background, see the Wolfi docs.

Quickstart

  • sg wolfi lock gitserver - update the .lock.json for gitserver with the latest set of package versions
  • sg wolfi image gitserver - build the gitserver base image

High-level Architecture

   file
  ┌──────────┐
  │          │
  │          │
  │   YAML   ├────────┐
  │          │        │          sg wolfi image <image>
  │          │        │                                            bazel target
  └─────┬────┘        │                   OR                      ┌─────────────────────┐
        │             │                                           │                     │
        │             │     bazel build //<image>/:base_image     │                     │
   sg wolfi lock      ├──────────────────────────────────────────►│     :base_image     │
   (manual step)      │                                           │                     │
        │             │                                           │                     │
   file │             │                                           └──────────┬──────────┘
  ┌─────▼─────┐       │                                                      │
  │           │       │                                                      │
  │           │       │                                                      │
  │ Lockfile  ├───────┘                                                      │
  │           │            ┌─────────────────────────────────────────────────┘
  │           │            │
  └───────────┘            │
                           │
                           │      bazel rule
                           │     ┌──────────────────────────────────────┐
                           │     │                                      │
                           │     │  oci_image(                          │
                           │     │                                      │
                           │     │    name = "image"                    │
                           │     │                                      │
                           └─────┼──► base = ":base_image"              │
    Bazel-genenarated            │                                      │
    binaries and      ───────────┼──► tars = ":tar_sourcegraph_binary"  │
    other resources              │                                      │
                                 │    [...]                             │
                                 │                                      │
                                 │  )                                   │
                                 │                                      │
                                 └──────────────────────────────────────┘