sourcegraph/dev/build-tracker
William Bezuidenhout ad3530166e
build-tracker: only account for terminal job states (#58834)
* more tests and additional logging

* Fix !finished bug causing all Jobs to be considered inprogress

* Remove inprogress since it is not a terminal state and we only consider
terminal states.
* Rework how status is determined.
* Add comments

* fix testcase

* Update dev/build-tracker/build/build.go
2023-12-08 15:40:47 +02:00
..
build build-tracker: only account for terminal job states (#58834) 2023-12-08 15:40:47 +02:00
config bazel: add depguard as a nogo linter (#50585) 2023-04-13 14:19:45 +02:00
notify build-tracker: only account for terminal job states (#58834) 2023-12-08 15:40:47 +02:00
util build-tracker: keep better track of jobs (#48378) 2023-03-14 06:43:26 +02:00
.gitignore feat(build-tracker): Adds a build-tracking service to provide better notifications about build failures (#39355) 2022-08-02 17:25:39 +00:00
BUILD.bazel ci: re-enable race detection (#52776) 2023-06-05 20:41:47 +02:00
build.sh Remove the custom built gitserver using Microsoft fork of git (continued) (#46121) 2023-01-05 16:02:07 +01:00
Dockerfile ci: enable bazel builds for docker images (#51241) 2023-04-28 10:41:13 +02:00
integration_test.go build-tracker: only account for terminal job states (#58834) 2023-12-08 15:40:47 +02:00
main_test.go build-tracker: only account for terminal job states (#58834) 2023-12-08 15:40:47 +02:00
main.go build-tracker: only account for terminal job states (#58834) 2023-12-08 15:40:47 +02:00
README.md buildtracker: fix readme (#58706) 2023-12-01 14:06:59 +02:00
server_test.go build-tracker: only account for terminal job states (#58834) 2023-12-08 15:40:47 +02:00

BUILD TRACKER

Build Tracker is a server that listens for build events from Buildkite and stores them in memory and sends notifications about builds if they've failed.

The server currently listens for two events:

  • build.finished
  • job.finished

For each job.finished event that is received, the corresponding build is updated with the job that has finished. On receipt of a build.finished event, the server will determine if the build has failed by going through all the contained jobs of the build. If one or more jobs have indeed failed, a notification will be sent over slack.

Deployment infrastructure

Build Tracker is deployed in the Buildkite kubernetes cluster of the Sourcegraph CI project on GCP. For more information on the deployment see infrastructure

Build

Execute the build.sh script which will build the docker container and push it to correct GCP registry. Once the image has been pushed the pod needs to be restarted so that it can pick up the new image!

Test

To run the tests execute go test .

Notification testing

To test the notifications that get sent over slack you can pass the flag -RunSlackIntegrationTest as part of your test invocation, with some required configuration:

export SLACK_TOKEN='my valid token'
export BUILDKITE_WEBHOOK_TOKEN='optional'
export GITHUB_TOKEN='optional'
go test . -RunSlackIntegrationTest

You can enable Slack client debugging by exporting the following environment variable BUILD_TRACKER_SLACK_DEBUG=1