Backport wolfi docs changes from sourcegraph/docs repo (#61874)

The dev/ docs were removed from the sourcegraph/docs repo in https://github.com/sourcegraph/docs/pull/253, so backport all changes to wolfi docs that were made

doc/dev/how-to/wolfi/add_update_images.md

@@ -8,24 +8,38 @@ Wolfi base images are built _from scratch_ using [apko](https://github.com/chain
 
 Base images are defined using an apko YAML configuration file, found under [wolfi-images](https://sourcegraph.com/github.com/sourcegraph/sourcegraph/-/tree/wolfi-images).
 
-These configuration files can be processed with apko, which will generate a base image. You can build these locally using `sg wolfi image <image-name>`.
+To make this YAML file declarative for Bazel, we generate a lockfile from the YAML similar to Go's `go.mod` or NPM's `package-lock.json`. This lockfile captures the exact versions and hashes of packages that will be included in the base image.
+
+These lockfiles files are processed by the rules_apko Bazel ruleset, which outputs a base image.
+
+You can see how this works locally:
+
+- Edit `wolfi-images/gitserver.yaml` to add a new package
+- Run `sg wolfi lock gitserver`
+  - Inspect the gitserver lockfile at `wolfi-images/gitserver.lock.json` to see that it now includes your new package, pinned at a specific version
+  - The lockfile may also include updated versions for other packages
+- Run `sg wolfi image gitserver`
+  - This uses Bazel to build the gitserver base image using the exact set of package versions defined in `gitserver.lock.json`.
+  - This wraps `bazel run //<image>:base_tarball`, but will automatically run `sg wolfi lock` whenever the YAML is changed.
 
 ## How to...
 
-### Update base images for a new release
+### Update base image dependencies
 
-Before each release, we should update the base images to ensure we include any updated packages and vulnerability fixes. To update the images:
+Periodically, we should update the base images to ensure we include any updated packages and vulnerability fixes. To update the images:
 
-- Review the [auto-update pull request](https://github.com/sourcegraph/sourcegraph/pulls?q=is:pr+head:wolfi-auto-update/main+is:open) opened by Buildkite, and merge it
+- Review the [auto-update pull requests](https://github.com/sourcegraph/sourcegraph/pulls?q=is:pr+head:wolfi-auto-update/+is:open+) opened by Buildkite, and merge them.
+
+Note: there are separate pull requests for `main` and the current release branch to avoid merge conflicts.
 
 #### Automation
 
 This process is automated by Buildkite, which runs a daily scheduled build to:
 
-- Rebuild Wolfi base images, pulling in any updated dependencies.
-- Push the updated base images to Docker Hub.
-- Update the base image hashes in `dev/oci_deps.bzl`.
-- Open, or update the already open pull request updating the hashes.
+- Run `sg wolfi lock` to update the base image lockfiles, pulling in the latest versions of each package.
+- Open, or update the previously-opened [pull request](https://github.com/sourcegraph/sourcegraph/pulls?q=is:pr+head:wolfi-auto-update/+is:open+).
+
+The automatic PR is updated daily with the expectation it will be merged before each release, rather than merged daily.
 
 To rerun the automation manually (perhaps to pick up a just-released package version or a change you made), open [Buildkite for sourcegraph/sourcegraph](https://buildkite.com/sourcegraph/sourcegraph) and choose New Build > Options > set Environment Variables to `WOLFI_BASE_REBUILD=true` and Create Build.
 
@@ -33,25 +47,20 @@ To rerun the automation manually (perhaps to pick up a just-released package ver
 
 If the automation fails and a manual update is needed, follow these steps:
 
-- Run [`wolfi-images/rebuild-images.sh`](https://sourcegraph.com/github.com/sourcegraph/sourcegraph@main/-/blob/wolfi-images/rebuild-images.sh) script, commit the updated YAML files, and merge to main.
-- Wait for the `main` branch's Buildkite run to complete.
-  - Buildkite will rebuild the base images and publish them to Dockerhub.
-- Run `sg wolfi update-hashes` locally to update the base image hashes in `dev/oci_deps.bzl`. Commit these changes and merge to `main`.
-  - This fetches the updated base image hashes from the images that were pushed to Dockerhub in the previous step.
-- Backport the PR that updated `dev/oci_deps.bzl` to the release branch.
+- Run `sg wolfi lock` to pull in the latest package versions in all images
+- Stage and commit changes using `git add wolfi-images/*.lock.json && git commit -m "Update base image lockfiles"`
+- Create a PR and merge to the target branch.
 
 ### Modify an existing base image
 
-To modify a base image to add packages, users, or directories:
+To modify a base image, for example to add packages, users, or directories:
 
-- Update its apko YAML configuration file, which can be found under [`wolfi-images/`](https://sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/wolfi-images/)
-- Build the image
-  - To build locally, use `sg wolfi image <image-name>`.
-  - To build on CI, push your changes and Buildkite will build your image and push it to our `us.gcr.io/sourcegraph-dev/` dev repository. Instructions for pulling this image will be shown at the top of the Buildkite page.
-- Test your changes by exec-ing into the image, or update `dev/oci_deps.bzl` to point at your dev base image and build the full image with Bazel.
-- Once happy, merge your changes and wait for the `main` branch's Buildkite run to complete.
-  - Buildkite will rebuild the base image and publish it to Dockerhub.
-- Run `sg wolfi update-hashes <image-name>` to update the hashes for the changed image in `dev/oci_deps.bzl`. Commit and merge these changes.
+- Update the image's apko YAML configuration file, which can be found under [`wolfi-images/`](https://sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/wolfi-images/)
+- To build the **base image** (all depencencies listed in the YAML file, *without* Sourcegraph binaries) run `sg wolfi image <image-name>`
+- To build the **full image** (all dependencies *and* Sourcegraph binaries):
+  - Run `sg wolfi lock <image-name>`
+  - Use Bazel to build the full image e.g. `bazel run //cmd/gitserver:image_tarball`
+- Once happy, commit any changes in `wolfi-images/` and push. Your changes changes to the image will be reflected in the Buildkite pipeline, and in the production images once merged to `main`.
 
 ### Create a new base image
 
@@ -63,10 +72,7 @@ Otherwise, you can create a new base image configuration file:
 - Add any required packages, users, directory structure, or metadata.
   - See [apko file format](https://github.com/chainguard-dev/apko/blob/main/docs/apko_file.md) for a full list of supported configuration.
   - See the other images under [`wolfi-images/`](https://sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/wolfi-images/) and [`chainguard-images/images`](https://github.com/chainguard-images/images/tree/main/images) for examples and best practices.
-- Build the image:
-  - To build locally, use `sg wolfi image <image-name>`.
-  - To build on CI, push your changes and Buildkite will build your image and push it to our `us.gcr.io/sourcegraph-dev/` dev repository.
-- Test your changes by exec-ing into the image, or update `dev/oci_deps.bzl` to point at your dev base image and build the full image with Bazel.
-- Commit your updated YAML file and merge it to main. Buildkite will build and publish your new image to Dockerhub.
-
-Once complete, treat the published image it as a standard Docker image, and add it to `dev/oci_deps.bzl`.
+- Build the base image locally using `sg wolfi image <image-name>`
+  - Test your changes by exec-ing into the image
+- Review an existing `BUILD.bazel` file like `cmd/gitserver/BUILD.bazel` for examples on how to declare an oci_image build target that uses your new base image.
+  - The key parts are to ensure the directory matches the YAML filename, calling `wolfi_base()`, and referencing `:wolfi_base_image`.

doc/dev/how-to/wolfi/add_update_packages.md

@@ -39,15 +39,15 @@ It's common to need to update a package to the most recent release in order to p
 - You should also reset the `package.epoch` field to 0 when updating the version.
   - The `version` is the version of the dependency being packaged, but the `epoch` is like the version of the _package itself_. It should be incremented whenever making changes, and reset to 0 when the dependency version increases.
 
-3. Optionally, build the package locally with `sg wolfi package <package-name>`.
+3. Optionally, build the package locally with `sg wolfi package <package-name>` and test by following the instructions in the output.
 
-4. Push your branch and create a PR. Open the Buildkite page for your build (`sg ci status --web`) - the pipeline will automatically rebuild any base images that use this package and show instructions for how to pull these locally for testing.
+4. Push your branch and open the Buildkite page `sg ci status --web`. Buildkite will build your package, push it to our dev package repository, and provide instructions at the top of the page for testing it locally.
 
-5. After validating the new package and base images, merge to `main`. This will add the updated packages to the [Sourcegraph package repository](#sourcegraph-package-repository) and push the updated base images to Dockerhub.
+5. After validating the new package and base images, merge to `main`. This will add the updated packages to the [Sourcegraph package repository](#sourcegraph-package-repository).
 
-6. Wait until the Buildkite pipeline for `main` has run - the next step depends on the images that this pipeline pushes to Dockerhub.
+6. Wait until the Buildkite pipeline for `main` has run - the next step depends on the packages having been pushed to our production package repo.
 
-7. Check out main, create a new branch, and run `sg wolfi update-hashes <image-name>` locally to update the base images digests in `dev/oci_deps.bzl`. Create another PR and merge to main. This ensures that the updated base images are used when building final images.
+7. Create a new PR from `main`, and run `sg wolfi lock`. This will pull in the new version of your packages in any image that uses them. Push, and merge.
 
 ### Create a new package