artin/sourcegraph
1
0
Fork 0
mirror of https://github.com/sourcegraph/sourcegraph.git synced 2026-02-06 14:51:44 +00:00
Code Issues Packages Projects Releases Wiki Activity

release-tool: refactor release config (#47711)

Browse Source 
Closes [#47557](https://github.com/sourcegraph/sourcegraph/issues/47557)

This is a somewhat gruesome PR to read, so sorry for that. I wanted to
keep this change atomic so it's easy to revert if we need.

So, I set out to automate the dates in the release config and realized
that fundamentally the architecture of the release tool was coupled to
this config in a way that was making life more difficult than it needed
to be.

This PR refactors the release config in the release tool. It uses an
entirely new schema, and for the most part automates every interaction
anyone will have with the config. The new schema is designed to do a few
things:
1. Deprecate the annoying cached version by specifying two sections of
the release config, both a definition and an active releases field.
Anything in the active releases is considered in progress.
2. Sets up some architecture to support multi-release functionality from
the release tool. There are guard rails right now since this isn't fully
supported.

Also,
* Automatically detect and suggest versions
* Automatically generate release dates from a specified date or current
time
* QOL changes to flow to the appropriate commands if the release config
is in the wrong state. For example it will prompt for input if the
config is empty for a given version for commands that require a valid
state
* Guardrails around every interaction I could find that causes problems
* New commands to interact with the release config (activate-release,
deactivate-release, util:previous-version)

## Usage

To set a release as active, either use the activate command directly
```
pnpm run release release:activate-release
```
or transitively through another dependent command.

All release commands that interact with the _current_ release will read
from the active release in the release config.
All release commands that interact with _future_ releases will read and
write to the release definitions in the release config.

To deactivate the release:
```
pnpm run release release:deactivate-release
```

Most of these steps are not explicitly mandatory, and are coupled into
other relevant commands (for example closing the release will deactivate
the release also).

To add new scheduled release definitions:
```
pnpm run release release:prepare
```

To remove scheduled releases:
```
pnpm run release release:remove
```

To determine previous versions:
```
pnpm run release util:previous-version 4.2.1
Getting previous version from: 4.2.1...
4.2.0
```

or

```
pnpm run release util:previous-version
Getting previous version...
4.4.2
```


## Test plan

I did a lot of manual testing since so many commands were impacted. Some
example output is below, and here is an example of a fake issue that was
generated from the release tool:
https://github.com/sourcegraph/sourcegraph/issues/47760

Example of how the input will flow if the config is not in the correct
state, including version suggestions:
```
> @sourcegraph/dev-release@0.0.1 release /Users/coury@sourcegraph.com/Documents/code/sourcegraph/dev/release
> ts-node --transpile-only ./src/main.ts "release:status"

No active releases are defined! Attempting to activate...
Next minor release: 4.5.0
Next patch release: 4.4.3
Enter the version to activate: 
```

```
> @sourcegraph/dev-release@0.0.1 release /Users/coury@sourcegraph.com/Documents/code/sourcegraph/dev/release
> ts-node --transpile-only ./src/main.ts "release:status"

No active releases are defined! Attempting to activate...
Next minor release: 4.5.0
Next patch release: 4.4.3
Enter the version to activate: 4.4.3
Attempting to detect previous version...
Detected previous version: 4.4.2
Release definition not found for: 4.4.3, enter release information.

Enter the release date (YYYY-MM-DD). Enter blank to use current date: 
Using current time: 2023-02-16T11:01:34.710-08:00
Enter the github username of the release captain: coury-clark
Enter the slack username of the release captain: coury
Version created:
{
  "codeFreezeDate": "2023-02-09T10:01:34.710-08:00",
  "securityApprovalDate": "2023-02-09T10:01:34.710-08:00",
  "releaseDate": "2023-02-16T10:01:34.710-08:00",
  "current": "4.4.3",
  "captainGitHubUsername": "coury-clark",
  "captainSlackUsername": "coury"
}
Release: 4.4.3 activated!
```


<!-- All pull requests REQUIRE a test plan:
https://docs.sourcegraph.com/dev/background-information/testing_principles
-->

## App preview:

-
[Web](https://sg-web-cclark-refactor-release-config.onrender.com/search)

Check out the [client app preview
documentation](https://docs.sourcegraph.com/dev/how-to/client_pr_previews)
to learn more.
This commit is contained in:
coury-clark 2023-02-17 11:00:42 -07:00 committed by GitHub
parent ff63a85382
commit dd1e512434
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
11 changed files with 610 additions and 366 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
dev/release/package.json
View File

@ -6,5 +6,9 @@
"scripts": {
"release": "ts-node --transpile-only ./src/main.ts",
"lint:js": "eslint --cache 'src/**/*.[jt]s?(x)'"
},
"dependencies": {
"@types/luxon": "^3.2.0",
"luxon": "^3.2.1"
}
}

54
dev/release/release-config.jsonc
View File

@ -1,39 +1,23 @@
/**
* Configuration for the release tool. All fields should align exactly with those in
* `src/config.ts`. To learn more about the release tool:
*
* pnpm run release help
*
* A quick sanity-check can be performed to make sure configuration is loaded correctly:
*
* pnpm run release _test:config
*/
{
// Captain information
// To get your slack username, navigate to Profile -> More -> Account settings -> Username (at the bottom) -> Expand
"captainSlackUsername": "keegan",
"captainGitHubUsername": "keegancsmith",
// Release versions
"previousRelease": "4.4.2",
"upcomingRelease": "4.5.0",
"oneWorkingWeekBeforeRelease": "15 February 2023 10:00 PST",
"threeWorkingDaysBeforeRelease": "17 February 2023 10:00 PST",
"releaseDate": "22 February 2023 10:00 PST",
"oneWorkingDayAfterRelease": "23 February 2023 10:00 PST",
"oneWorkingWeekAfterRelease": "1 March 2023 10:00 PST",
// Channel where messages from the tooling are posted
"slackAnnounceChannel": "announce-engineering",
// Email for preparing calendar events
"metadata": {
"teamEmail": "team@sourcegraph.com",
// Enable dry-running for certain features - useful for testing or sanity-checking.
//
// For example, `dryRun.changesets` prints changes generated by `release:stage`
// instead of pushing them to GitHub.
"dryRun": {
"tags": false,
"changesets": false,
"trackingIssues": false,
"calendar": false,
"slack": false
"slackAnnounceChannel": "announce-engineering"
},
"scheduledReleases": {
"4.5.0": {
"codeFreezeDate": "2023-02-15T10:00:00.000-08:00",
"securityApprovalDate": "2023-02-15T10:00:00.000-08:00",
"releaseDate": "2023-02-22T10:00:00.000-08:00",
"current": "4.5.0",
"captainGitHubUsername": "coury-clark",
"captainSlackUsername": "coury"
}
},
"dryRun": {
"tags": false,
"changesets": false,
"trackingIssues": false,
"calendar": false,
"slack": false
}
}

260
dev/release/src/config.ts
View File

@ -1,9 +1,15 @@
import { readFileSync, unlinkSync } from 'fs'
import { readFileSync, writeFileSync } from 'fs'
import chalk from 'chalk'
import { parse as parseJSONC } from 'jsonc-parser'
import { DateTime } from 'luxon'
import * as semver from 'semver'
import { SemVer } from 'semver'
import { cacheFolder, readLine, getWeekNumber } from './util'
import { getPreviousVersion } from './git'
import { retryInput } from './util'
const releaseConfigPath = 'release-config.jsonc'
/**
* Release configuration file format
@ -34,63 +40,213 @@ export interface Config {
}
}
/**
* Default path of JSONC containing release configuration.
*/
const configPath = 'release-config.jsonc'
/**
* Loads configuration from predefined path. It does not do any special validation.
*/
export function loadConfig(): Config {
return parseJSONC(readFileSync(configPath).toString()) as Config
export async function getActiveRelease(config: ReleaseConfig): Promise<ActiveRelease> {
if (!config.in_progress || config.in_progress.releases.length === 0) {
console.log(chalk.yellow('No active releases are defined! Attempting to activate...'))
await activateRelease(config)
}
if (!config.in_progress) {
throw new Error('unable to activate a release!')
}
if (config.in_progress.releases.length > 1) {
throw new Error(
chalk.red(
'The release config has multiple versions activated. This feature is not yet supported by the release tool! Please activate only a single release.'
)
)
}
const rel = config.in_progress.releases[0]
const def = config.scheduledReleases[rel.version]
const version = new SemVer(rel.version)
return {
version,
previous: new SemVer(rel.previous),
...(def as ReleaseDates),
...(def as ReleaseCaptainInformation),
branch: `${version.minor}.${version.minor}`,
}
}
/**
* Convenience function for getting relevant configured releases as semver.SemVer
*
* It prompts for a confirmation of the `upcomingRelease` that is cached for a week.
*/
export async function releaseVersions(config: Config): Promise<{
previous: semver.SemVer
upcoming: semver.SemVer
}> {
const parseOptions: semver.Options = { loose: false }
const parsedPrevious = semver.parse(config.previousRelease, parseOptions)
if (!parsedPrevious) {
throw new Error(`config.previousRelease '${config.previousRelease}' is not valid semver`)
export function loadReleaseConfig(): ReleaseConfig {
return parseJSONC(readFileSync(releaseConfigPath).toString()) as ReleaseConfig
}
export function saveReleaseConfig(config: ReleaseConfig): void {
writeFileSync(releaseConfigPath, JSON.stringify(config, null, 2))
}
export function newRelease(
version: SemVer,
releaseDate: DateTime,
captainGithub: string,
captainSlack: string
): ScheduledReleaseDefinition {
return {
...releaseDates(releaseDate),
current: version.version,
captainGitHubUsername: captainGithub,
captainSlackUsername: captainSlack,
}
const parsedUpcoming = semver.parse(config.upcomingRelease, parseOptions)
if (!parsedUpcoming) {
throw new Error(`config.upcomingRelease '${config.upcomingRelease}' is not valid semver`)
}
export async function newReleaseFromInput(versionOverride?: SemVer): Promise<ScheduledReleaseDefinition> {
let version = versionOverride
if (!version) {
version = await selectVersionWithSuggestion('Enter the desired version number')
}
// Verify the configured upcoming release. The response is cached and expires in a
// week, after which the captain is required to confirm again.
const now = new Date()
const cachedVersionResponse = `${cacheFolder}/current_release_${now.getUTCFullYear()}_${getWeekNumber(now)}.txt`
const confirmVersion = await readLine(
`Please confirm the upcoming release version configured in '${configPath}' (currently '${config.upcomingRelease}') by entering it again: `,
cachedVersionResponse
const releaseDateStr = await retryInput(
'Enter the release date (YYYY-MM-DD). Enter blank to use current date: ',
val => {
if (val && /^\d{4}-\d{2}-\d{2}$/.test(val)) {
return true
}
// this will return false if the input doesn't match the regexp above but does exist, allowing blank input to still be valid
return !val
},
'invalid date, expected format YYYY-MM-DD'
)
const parsedConfirmed = semver.parse(confirmVersion, parseOptions)
let error = ''
if (!parsedConfirmed) {
error = `Provided version '${confirmVersion}' is not valid semver`
} else if (semver.neq(parsedConfirmed, parsedUpcoming)) {
error = `Provided version '${confirmVersion}' and config.upcomingRelease '${config.upcomingRelease}' do not match - please update the release configuration at '${configPath}' and try again`
let releaseTime: DateTime
if (!releaseDateStr) {
releaseTime = DateTime.now().setZone('America/Los_Angeles')
console.log(chalk.blue(`Using current time: ${releaseTime.toString()}`))
} else {
releaseTime = DateTime.fromISO(releaseDateStr, { zone: 'America/Los_Angeles' })
}
// If error, abort and remove the cached response (since it is invalid anyway)
if (error !== '') {
unlinkSync(cachedVersionResponse)
throw new Error(error)
}
const captainGithubUsername = await retryInput('Enter the github username of the release captain: ', val => !!val)
const captainSlackUsername = await retryInput('Enter the slack username of the release captain: ', val => !!val)
const versions = {
previous: parsedPrevious,
upcoming: parsedUpcoming,
}
console.log(`Using versions: { upcoming: ${versions.upcoming.format()}, previous: ${versions.previous.format()} }`)
return versions
const rel = newRelease(version, releaseTime, captainGithubUsername, captainSlackUsername)
console.log(chalk.green('Version created:'))
console.log(chalk.green(JSON.stringify(rel, null, 2)))
return rel
}
function releaseDates(releaseDate: DateTime): ReleaseDates {
releaseDate = releaseDate.set({ hour: 10 })
return {
codeFreezeDate: releaseDate.plus({ days: -7 }).toString(),
securityApprovalDate: releaseDate.plus({ days: -7 }).toString(),
releaseDate: releaseDate.toString(),
}
}
export function addScheduledRelease(config: ReleaseConfig, release: ScheduledReleaseDefinition): ReleaseConfig {
config.scheduledReleases[release.current] = release
return config
}
export function removeScheduledRelease(config: ReleaseConfig, version: string): ReleaseConfig {
delete config.scheduledReleases[version]
return config
}
export interface ReleaseDates {
releaseDate: string
codeFreezeDate: string
securityApprovalDate: string
}
export interface ActiveRelease extends ReleaseCaptainInformation, ReleaseDates {
version: SemVer
previous: SemVer
branch: string
}
export interface ActiveReleaseDefinition {
version: string
previous: string
}
export interface ReleaseCaptainInformation {
captainSlackUsername: string
captainGitHubUsername: string
}
export interface InProgress extends ReleaseCaptainInformation {
releases: ActiveReleaseDefinition[]
}
export interface ReleaseConfig {
metadata: {
teamEmail: string
slackAnnounceChannel: string
}
scheduledReleases: {
[version: string]: ScheduledReleaseDefinition
}
in_progress?: InProgress
dryRun: {
tags?: boolean
changesets?: boolean
trackingIssues?: boolean
slack?: boolean
calendar?: boolean
}
}
export interface ScheduledReleaseDefinition extends ReleaseDates, ReleaseCaptainInformation {
current: string
}
// Prompt a user for input and activate the given release version if possible. Will redirect to release creation input if
// the version isn't defined.
export async function activateRelease(config: ReleaseConfig): Promise<void> {
const next = await selectVersionWithSuggestion('Enter the version to activate')
console.log('Attempting to detect previous version...')
const previous = getPreviousVersion(next)
console.log(chalk.blue(`Detected previous version: ${previous.version}`))
const scheduled = await getScheduledReleaseWithInput(config, next)
config.in_progress = {
captainGitHubUsername: scheduled.captainGitHubUsername,
captainSlackUsername: scheduled.captainSlackUsername,
releases: [{ version: next.version, previous: previous.version }],
}
saveReleaseConfig(config)
console.log(chalk.green(`Release: ${next.version} activated!`))
}
export function deactivateAllReleases(config: ReleaseConfig): void {
delete config.in_progress
saveReleaseConfig(config)
}
// Prompt a user for a major / minor version input with automation suggestion by adding a minor version to the previous version.
async function selectVersionWithSuggestion(prompt: string): Promise<SemVer> {
const probablyMinor = getPreviousVersion().inc('minor')
const probablyPatch = getPreviousVersion().inc('patch')
const input = await retryInput(
`Next minor release: ${probablyMinor.version}\nNext patch release: ${probablyPatch.version}\n${chalk.blue(
prompt
)}: `,
val => {
const version = semver.parse(val)
return !!version
}
)
return new SemVer(input)
}
// Prompt a user for a release definition input, and redirect to creation input if it doesn't exist.
export async function getReleaseDefinition(config: ReleaseConfig): Promise<ScheduledReleaseDefinition> {
const next = await selectVersionWithSuggestion('Enter the version number to select')
return getScheduledReleaseWithInput(config, next)
}
// Helper function to get a release definition from the release config, redirecting to creation input if it doesn't exist.
async function getScheduledReleaseWithInput(
config: ReleaseConfig,
releaseVersion: SemVer
): Promise<ScheduledReleaseDefinition> {
let scheduled = config.scheduledReleases[releaseVersion.version]
if (!scheduled) {
console.log(
chalk.yellow(`Release definition not found for: ${releaseVersion.version}, enter release information.\n`)
)
scheduled = await newReleaseFromInput(releaseVersion)
addScheduledRelease(config, scheduled)
}
return scheduled
}

48
dev/release/src/git.ts Normal file
View File

@ -0,0 +1,48 @@
import execa from 'execa'
import { SemVer } from 'semver'
import * as semver from 'semver'
import { localSourcegraphRepo } from './github'
export function getTags(workdir: string, prefix?: string): string[] {
execa.sync('git', ['fetch', '--tags'], { cwd: workdir })
return execa
.sync('git', ['--no-pager', 'tag', '-l', `${prefix}`, '--sort=v:refname'], { cwd: workdir })
.stdout.split('\n')
}
export function getCandidateTags(workdir: string, version: string): string[] {
return getTags(workdir, `v${version}-rc*`)
}
export function getReleaseTags(workdir: string): string[] {
const raw = getTags(workdir, 'v[0-9]*.[0-9]*.[0-9]*')
// since tags are globbed they can overmatch when we just want pure release tags
return raw.filter(tag => tag.match('[0-9]+\\.[0-9]+\\.[0-9]+$'))
}
// Returns the version tagged in the repository previous to a provided input version. If no input version it will
// simply return the highest version found in the repository.
export function getPreviousVersion(version?: SemVer): SemVer {
const lowest = new SemVer('0.0.1')
const tags = getReleaseTags(localSourcegraphRepo)
if (tags.length === 0) {
return lowest
}
if (!version) {
return new SemVer(tags[tags.length - 1])
}
for (
let reallyLongVariableNameBecauseESLintRulesAreSilly = tags.length - 1;
reallyLongVariableNameBecauseESLintRulesAreSilly >= 0;
reallyLongVariableNameBecauseESLintRulesAreSilly--
) {
const tag = tags[reallyLongVariableNameBecauseESLintRulesAreSilly]
const temp = semver.parse(tag)
if (temp && temp.compare(version) === -1) {
return temp
}
}
return lowest
}

63
dev/release/src/github.ts
View File

@ -1,4 +1,4 @@
import { mkdtemp as original_mkdtemp, readFileSync, existsSync } from 'fs'
import { existsSync, mkdtemp as original_mkdtemp, readFileSync } from 'fs'
import * as os from 'os'
import * as path from 'path'
import { promisify } from 'util'
@ -9,7 +9,8 @@ import execa from 'execa'
import fetch from 'node-fetch'
import * as semver from 'semver'
import { readLine, formatDate, timezoneLink, cacheFolder, changelogURL, getContainerRegistryCredential } from './util'
import { cacheFolder, changelogURL, formatDate, getContainerRegistryCredential, readLine, timezoneLink } from './util'
const mkdtemp = promisify(original_mkdtemp)
let githubPAT: string
@ -88,21 +89,17 @@ interface IssueTemplateArguments {
*/
version: semver.SemVer
/**
* Available as `$ONE_WORKING_WEEK_BEFORE_RELEASE`
* Available as `$SECURITY_REVIEW_DATE`
*/
oneWorkingWeekBeforeRelease: Date
securityReviewDate: Date
/**
* Available as `$ONE_WORKING_DAY_BEFORE_RELEASE`
* Available as `$CODE_FREEZE_DATE`
*/
threeWorkingDaysBeforeRelease: Date
codeFreezeDate: Date
/**
* Available as `$RELEASE_DATE`
*/
releaseDate: Date
/**
* Available as `$ONE_WORKING_DAY_AFTER_RELEASE`
*/
oneWorkingDayAfterRelease: Date
}
/**
@ -143,13 +140,7 @@ function dateMarkdown(date: Date, name: string): string {
async function execTemplate(
octokit: Octokit,
template: IssueTemplate,
{
version,
oneWorkingWeekBeforeRelease,
threeWorkingDaysBeforeRelease,
releaseDate,
oneWorkingDayAfterRelease,
}: IssueTemplateArguments
{ version, securityReviewDate, codeFreezeDate, releaseDate }: IssueTemplateArguments
): Promise<string> {
console.log(`Preparing issue from ${JSON.stringify(template)}`)
const name = releaseName(version)
@ -158,19 +149,9 @@ async function execTemplate(
.replace(/\$MAJOR/g, version.major.toString())
.replace(/\$MINOR/g, version.minor.toString())
.replace(/\$PATCH/g, version.patch.toString())
.replace(
/\$ONE_WORKING_WEEK_BEFORE_RELEASE/g,
dateMarkdown(oneWorkingWeekBeforeRelease, `One working week before ${name} release`)
)
.replace(
/\$THREE_WORKING_DAY_BEFORE_RELEASE/g,
dateMarkdown(threeWorkingDaysBeforeRelease, `Three working days before ${name} release`)
)
.replace(/\$SECURITY_REVIEW_DATE/g, dateMarkdown(securityReviewDate, `One working week before ${name} release`))
.replace(/\$CODE_FREEZE_DATE/g, dateMarkdown(codeFreezeDate, `Three working days before ${name} release`))
.replace(/\$RELEASE_DATE/g, dateMarkdown(releaseDate, `${name} release date`))
.replace(
/\$ONE_WORKING_DAY_AFTER_RELEASE/g,
dateMarkdown(oneWorkingDayAfterRelease, `One working day after ${name} release`)
)
}
interface MaybeIssue {
@ -189,17 +170,15 @@ export async function ensureTrackingIssues({
version,
assignees,
releaseDate,
oneWorkingWeekBeforeRelease,
threeWorkingDaysBeforeRelease,
oneWorkingDayAfterRelease,
securityReviewDate,
codeFreezeDate,
dryRun,
}: {
version: semver.SemVer
assignees: string[]
releaseDate: Date
oneWorkingWeekBeforeRelease: Date
threeWorkingDaysBeforeRelease: Date
oneWorkingDayAfterRelease: Date
securityReviewDate: Date
codeFreezeDate: Date
dryRun: boolean
}): Promise<MaybeIssue[]> {
const octokit = await getAuthenticatedGitHubClient()
@ -234,9 +213,8 @@ export async function ensureTrackingIssues({
const body = await execTemplate(octokit, template, {
version,
releaseDate,
oneWorkingWeekBeforeRelease,
threeWorkingDaysBeforeRelease,
oneWorkingDayAfterRelease,
securityReviewDate,
codeFreezeDate,
})
const issue = await ensureIssue(
octokit,
@ -735,12 +713,3 @@ export async function closeTrackingIssue(version: semver.SemVer): Promise<void>
await closeIssue(octokit, previousIssue)
}
}
export function getTags(workdir: string, prefix: string): string[] {
execa.sync('git', ['fetch', '--tags'], { cwd: workdir })
return execa.sync('git', ['--no-pager', 'tag', '-l', `${prefix}`], { cwd: workdir }).stdout.split('\t')
}
export function getCandidateTags(workdir: string, version: string): string[] {
return getTags(workdir, `v${version}-rc*`)
}

4
dev/release/src/main.ts
View File

@ -1,4 +1,4 @@
import { loadConfig } from './config'
import { loadReleaseConfig } from './config'
import { runStep, StepID } from './release'
import { ensureMainBranchUpToDate } from './util'
@ -6,7 +6,7 @@ import { ensureMainBranchUpToDate } from './util'
* Release captain automation
*/
async function main(): Promise<void> {
const config = loadConfig()
const config = loadReleaseConfig()
const args = process.argv.slice(2)
if (args.length === 0) {
await runStep(config, 'help')

450
dev/release/src/release.ts
View File

@ -1,44 +1,57 @@
import { readFileSync, rmdirSync, writeFileSync } from 'fs'
import * as path from 'path'
import chalk from 'chalk'
import commandExists from 'command-exists'
import { addMinutes } from 'date-fns'
import execa from 'execa'
import { SemVer } from 'semver'
import * as batchChanges from './batchChanges'
import * as changelog from './changelog'
import { Config, releaseVersions } from './config'
import {
getAuthenticatedGitHubClient,
listIssues,
getTrackingIssue,
activateRelease,
addScheduledRelease,
loadReleaseConfig,
newReleaseFromInput,
ReleaseConfig,
getActiveRelease,
removeScheduledRelease,
saveReleaseConfig,
getReleaseDefinition,
deactivateAllReleases,
} from './config'
import { getCandidateTags, getPreviousVersion } from './git'
import {
cloneRepo,
closeTrackingIssue,
commentOnIssue,
createChangesets,
CreatedChangeset,
createLatestRelease,
createTag,
ensureTrackingIssues,
closeTrackingIssue,
releaseName,
commentOnIssue,
queryIssues,
getAuthenticatedGitHubClient,
getTrackingIssue,
IssueLabel,
createLatestRelease,
cloneRepo,
getCandidateTags,
listIssues,
localSourcegraphRepo,
queryIssues,
releaseName,
} from './github'
import { ensureEvent, getClient, EventOptions, calendarTime } from './google-calendar'
import { calendarTime, ensureEvent, EventOptions, getClient } from './google-calendar'
import { postMessage, slackURL } from './slack'
import {
cacheFolder,
formatDate,
timezoneLink,
ensureDocker,
changelogURL,
ensureDocker,
ensureReleaseBranchUpToDate,
ensureSrcCliEndpoint,
ensureSrcCliUpToDate,
getLatestTag,
formatDate,
getAllUpgradeGuides,
getLatestTag,
timezoneLink,
updateUpgradeGuides,
verifyWithInput,
} from './util'
@ -64,8 +77,13 @@ export type StepID =
| 'release:announce'
| 'release:close'
| 'release:multi-version-bake'
| 'release:prepare'
| 'release:remove'
| 'release:activate-release'
| 'release:deactivate-release'
// util
| 'util:clear-cache'
| 'util:previous-version'
// testing
| '_test:google-calendar'
| '_test:slack'
@ -79,7 +97,7 @@ export type StepID =
/**
* Runs given release step with the provided configuration and arguments.
*/
export async function runStep(config: Config, step: StepID, ...args: string[]): Promise<void> {
export async function runStep(config: ReleaseConfig, step: StepID, ...args: string[]): Promise<void> {
if (!steps.map(({ id }) => id as string).includes(step)) {
throw new Error(`Unrecognized step ${JSON.stringify(step)}`)
}
@ -97,7 +115,9 @@ export async function runStep(config: Config, step: StepID, ...args: string[]):
interface Step {
id: StepID
description: string
run?: ((config: Config, ...args: string[]) => Promise<void>) | ((config: Config, ...args: string[]) => void)
run?:
| ((config: ReleaseConfig, ...args: string[]) => Promise<void>)
| ((config: ReleaseConfig, ...args: string[]) => void)
argNames?: string[]
}
@ -132,48 +152,32 @@ const steps: Step[] = [
id: 'tracking:timeline',
description: 'Generate a set of Google Calendar events for a MAJOR.MINOR release',
run: async config => {
const { upcoming: release } = await releaseVersions(config)
const name = releaseName(release)
const next = await getReleaseDefinition(config)
const name = releaseName(new SemVer(next.current))
const events: EventOptions[] = [
{
title: `Security Team to Review Release Container Image Scans ${name}`,
description: '(This is not an actual event to attend, just a calendar marker.)',
anyoneCanAddSelf: true,
attendees: [config.teamEmail],
attendees: [config.metadata.teamEmail],
transparency: 'transparent',
...calendarTime(config.oneWorkingWeekBeforeRelease),
...calendarTime(next.securityApprovalDate),
},
{
title: `Cut Sourcegraph ${name}`,
description: '(This is not an actual event to attend, just a calendar marker.)',
anyoneCanAddSelf: true,
attendees: [config.teamEmail],
attendees: [config.metadata.teamEmail],
transparency: 'transparent',
...calendarTime(config.threeWorkingDaysBeforeRelease),
...calendarTime(next.codeFreezeDate),
},
{
title: `Release Sourcegraph ${name}`,
description: '(This is not an actual event to attend, just a calendar marker.)',
anyoneCanAddSelf: true,
attendees: [config.teamEmail],
attendees: [config.metadata.teamEmail],
transparency: 'transparent',
...calendarTime(config.releaseDate),
},
{
title: `Start deploying Sourcegraph ${name} to Cloud instances`,
description: '(This is not an actual event to attend, just a calendar marker.)',
anyoneCanAddSelf: true,
attendees: [config.teamEmail],
transparency: 'transparent',
...calendarTime(config.oneWorkingDayAfterRelease),
},
{
title: `All Cloud instances upgraded to Sourcegraph ${name}`,
description: '(This is not an actual event to attend, just a calendar marker.)',
anyoneCanAddSelf: true,
attendees: [config.teamEmail],
transparency: 'transparent',
...calendarTime(config.oneWorkingWeekAfterRelease),
...calendarTime(next.releaseDate),
},
]
@ -190,53 +194,43 @@ const steps: Step[] = [
},
{
id: 'tracking:issues',
description: 'Generate GitHub tracking issue for the configured release',
run: async (config: Config) => {
const {
releaseDate,
captainGitHubUsername,
oneWorkingWeekBeforeRelease,
threeWorkingDaysBeforeRelease,
oneWorkingDayAfterRelease,
captainSlackUsername,
slackAnnounceChannel,
dryRun,
} = config
const { upcoming: release } = await releaseVersions(config)
const date = new Date(releaseDate)
description: 'Generate GitHub tracking issues for a release',
run: async (config: ReleaseConfig) => {
const next = await getReleaseDefinition(config)
const version = new SemVer(next.current)
const date = new Date(next.releaseDate)
// Create issue
const trackingIssues = await ensureTrackingIssues({
version: release,
assignees: [captainGitHubUsername],
version,
assignees: [next.captainGitHubUsername],
releaseDate: date,
oneWorkingWeekBeforeRelease: new Date(oneWorkingWeekBeforeRelease),
threeWorkingDaysBeforeRelease: new Date(threeWorkingDaysBeforeRelease),
oneWorkingDayAfterRelease: new Date(oneWorkingDayAfterRelease),
dryRun: dryRun.trackingIssues || false,
securityReviewDate: new Date(next.securityApprovalDate),
codeFreezeDate: new Date(next.codeFreezeDate),
dryRun: config.dryRun.trackingIssues || false,
})
console.log('Rendered tracking issues', trackingIssues)
// If at least one issue was created, post to Slack
if (trackingIssues.find(({ created }) => created)) {
const name = releaseName(release)
const name = releaseName(version)
const releaseDateString = slackURL(formatDate(date), timezoneLink(date, `${name} release`))
let annoncement = `:mega: *${name} release*
:captain: Release captain: @${captainSlackUsername}
:captain: Release captain: @${next.captainSlackUsername}
:spiral_calendar_pad: Scheduled for: ${releaseDateString}
:pencil: Tracking issues:
${trackingIssues.map(index => `- ${slackURL(index.title, index.url)}`).join('\n')}`
if (release.patch !== 0) {
const patchRequestTemplate = `https://github.com/sourcegraph/sourcegraph/issues/new?assignees=&labels=team%2Fdistribution&template=request_patch_release.md&title=${release.version}%3A+`
if (version.patch !== 0) {
const patchRequestTemplate = `https://github.com/sourcegraph/sourcegraph/issues/new?assignees=&labels=team%2Fdistribution&template=request_patch_release.md&title=${version.version}%3A+`
annoncement += `\n\nIf you have changes that should go into this patch release, ${slackURL(
'please *file a patch request issue*',
patchRequestTemplate
)}, or it will not be included.`
}
if (!dryRun.slack) {
await postMessage(annoncement, slackAnnounceChannel)
console.log(`Posted to Slack channel ${slackAnnounceChannel}`)
if (!config.dryRun.slack) {
await postMessage(annoncement, config.metadata.slackAnnounceChannel)
console.log(`Posted to Slack channel ${config.metadata.slackAnnounceChannel}`)
}
} else {
console.log('No tracking issues were created, skipping Slack announcement')
@ -248,8 +242,8 @@ ${trackingIssues.map(index => `- ${slackURL(index.title, index.url)}`).join('\n'
description: 'Generate pull requests to perform a changelog cut for branch cut',
argNames: ['changelogFile'],
run: async (config, changelogFile = 'CHANGELOG.md') => {
const { upcoming: release } = await releaseVersions(config)
const prMessage = `changelog: cut sourcegraph@${release.version}`
const upcoming = await getActiveRelease(config)
const prMessage = `changelog: cut sourcegraph@${upcoming.version.version}`
const pullRequest = await createChangesets({
requiredCommands: [],
changes: [
@ -257,17 +251,17 @@ ${trackingIssues.map(index => `- ${slackURL(index.title, index.url)}`).join('\n'
owner: 'sourcegraph',
repo: 'sourcegraph',
base: 'main',
head: `changelog-${release.version}`,
head: `changelog-${upcoming.version.version}`,
title: prMessage,
commitMessage: prMessage + '\n\n ## Test plan\n\nn/a',
edits: [
(directory: string) => {
console.log(`Updating '${changelogFile} for ${release.format()}'`)
console.log(`Updating '${changelogFile} for ${upcoming.version.format()}'`)
const changelogPath = path.join(directory, changelogFile)
let changelogContents = readFileSync(changelogPath).toString()
// Convert 'unreleased' to a release
const releaseHeader = `## ${release.format()}`
const releaseHeader = `## ${upcoming.version.format()}`
const unreleasedHeader = '## Unreleased'
changelogContents = changelogContents.replace(unreleasedHeader, releaseHeader)
@ -286,20 +280,20 @@ ${trackingIssues.map(index => `- ${slackURL(index.title, index.url)}`).join('\n'
owner: 'sourcegraph',
repo: 'deploy-sourcegraph-helm',
base: 'main',
head: `changelog-${release.version}`,
head: `changelog-${upcoming.version.version}`,
title: prMessage,
commitMessage: prMessage,
body: prMessage + '\n\n ## Test plan\n\nn/a',
edits: [
(directory: string) => {
console.log(`Updating '${changelogFile} for ${release.format()}'`)
console.log(`Updating '${changelogFile} for ${upcoming.version.format()}'`)
const changelogPath = path.join(directory, 'charts', 'sourcegraph', changelogFile)
let changelogContents = readFileSync(changelogPath).toString()
// Convert 'unreleased' to a release
const releaseHeader = `## ${release.format()}`
const releaseHeader = `## ${upcoming.version.format()}`
const releaseUpdate =
releaseHeader + `\n\n- Sourcegraph ${release.format()} is now available\n`
releaseHeader + `\n\n- Sourcegraph ${upcoming.version.format()} is now available\n`
const unreleasedHeader = '## Unreleased\n'
changelogContents = changelogContents.replace(unreleasedHeader, releaseUpdate)
@ -327,21 +321,22 @@ ${trackingIssues.map(index => `- ${slackURL(index.title, index.url)}`).join('\n'
id: 'release:branch-cut',
description: 'Create release branch',
run: async config => {
const { upcoming: release } = await releaseVersions(config)
const branch = `${release.major}.${release.minor}`
const release = await getActiveRelease(config)
let message: string
// notify cs team on patch release cut
if (release.patch !== 0) {
message = `:mega: *${release.version}* branch has been cut cc: @cs`
if (release.version.patch !== 0) {
message = `:mega: *${release.version.version}* branch has been cut cc: @cs`
} else {
message = `:mega: *${release.version}* branch has been cut.`
message = `:mega: *${release.version.version}* branch has been cut.`
}
try {
// Create and push new release branch from changelog commit
await execa('git', ['branch', branch])
await execa('git', ['push', 'origin', branch])
await postMessage(message, config.slackAnnounceChannel)
console.log(`To check the status of the branch, run:\nsg ci status -branch ${release.version} --wait\n`)
await execa('git', ['branch', release.branch])
await execa('git', ['push', 'origin', release.branch])
await postMessage(message, config.metadata.slackAnnounceChannel)
console.log(
`To check the status of the branch, run:\nsg ci status -branch ${release.version.version} --wait\n`
)
} catch (error) {
console.error('Failed to create release branch', error)
}
@ -352,11 +347,13 @@ ${trackingIssues.map(index => `- ${slackURL(index.title, index.url)}`).join('\n'
description: 'Post a message in Slack summarizing the progress of a release',
run: async config => {
const githubClient = await getAuthenticatedGitHubClient()
const { upcoming: release } = await releaseVersions(config)
const release = await getActiveRelease(config)
const trackingIssue = await getTrackingIssue(githubClient, release)
const trackingIssue = await getTrackingIssue(githubClient, release.version)
if (!trackingIssue) {
throw new Error(`Tracking issue for version ${release.version} not found - has it been created yet?`)
throw new Error(
`Tracking issue for version ${release.version.version} not found - has it been created yet?`
)
}
const latestTag = (await getLatestTag('sourcegraph', 'sourcegraph')).toString()
const latestBuildURL = `https://buildkite.com/sourcegraph/sourcegraph/builds?branch=${latestTag}`
@ -373,13 +370,15 @@ ${trackingIssues.map(index => `- ${slackURL(index.title, index.url)}`).join('\n'
: `are ${blockingIssues.length} release-blocking issues`
}`
const message = `:mega: *${release.version} Release Status Update*
const message = `:mega: *${release.version.version} Release Status Update*
* Tracking issue: ${trackingIssue.url}
* ${blockingMessage}: ${blockingIssuesURL}
* ${latestBuildMessage}`
if (!config.dryRun.slack) {
await postMessage(message, config.slackAnnounceChannel)
await postMessage(message, config.metadata.slackAnnounceChannel)
} else {
console.log(chalk.green('Dry run: ' + message))
}
},
},
@ -387,18 +386,20 @@ ${trackingIssues.map(index => `- ${slackURL(index.title, index.url)}`).join('\n'
id: 'release:create-candidate',
description: 'Generate the Nth release candidate. Set <candidate> to "final" to generate a final release',
run: async config => {
const { upcoming: release } = await releaseVersions(config)
const branch = `${release.major}.${release.minor}`
ensureReleaseBranchUpToDate(branch)
const release = await getActiveRelease(config)
ensureReleaseBranchUpToDate(release.branch)
const owner = 'sourcegraph'
const repo = 'sourcegraph'
try {
const client = await getAuthenticatedGitHubClient()
const { workdir } = await cloneRepo(client, owner, repo, { revision: branch, revisionMustExist: true })
const { workdir } = await cloneRepo(client, owner, repo, {
revision: release.branch,
revisionMustExist: true,
})
const tags = getCandidateTags(workdir, release.version)
const tags = getCandidateTags(workdir, release.version.version)
let nextCandidate = 1
for (const tag of tags) {
const num = parseInt(tag.slice(-1), 10)
@ -406,7 +407,7 @@ ${trackingIssues.map(index => `- ${slackURL(index.title, index.url)}`).join('\n'
nextCandidate = num + 1
}
}
const tag = `v${release.version}-rc.${nextCandidate}`
const tag = `v${release.version.version}-rc.${nextCandidate}`
console.log(`Detected next candidate: ${nextCandidate}, attempting to create tag: ${tag}`)
await createTag(
@ -415,7 +416,7 @@ ${trackingIssues.map(index => `- ${slackURL(index.title, index.url)}`).join('\n'
{
owner,
repo,
branch,
branch: release.branch,
tag,
},
config.dryRun.tags || false
@ -432,13 +433,12 @@ ${trackingIssues.map(index => `- ${slackURL(index.title, index.url)}`).join('\n'
'Promote a release candidate to release build. Specify the full candidate tag to promote the tagged commit to release.',
argNames: ['candidate'],
run: async (config, candidate) => {
const { upcoming: release } = await releaseVersions(config)
const releaseBranch = `${release.major}.${release.minor}`
ensureReleaseBranchUpToDate(releaseBranch)
const release = await getActiveRelease(config)
ensureReleaseBranchUpToDate(release.branch)
const warnMsg =
'Verify the provided tag is correct to promote to release. Note: it is very unusual to require a non-standard tag to promote to release, proceed with caution.'
const exampleTag = `v${release.version}-rc.1`
const exampleTag = `v${release.version.version}-rc.1`
if (!candidate) {
throw new Error(
`Candidate tag is a required argument. This should be the git tag of the commit to promote to release (ex.${exampleTag}`
@ -447,16 +447,16 @@ ${trackingIssues.map(index => `- ${slackURL(index.title, index.url)}`).join('\n'
await verifyWithInput(
`Warning!\nCandidate tag: ${candidate} does not match the standard convention (ex. ${exampleTag}). ${warnMsg}`
)
} else if (!candidate.match(`${release.version}-rc\\.\\d`)) {
} else if (!candidate.match(`${release.version.version}-rc\\.\\d`)) {
await verifyWithInput(
`Warning!\nCandidate tag: ${candidate} does not match the expected version ${release.version} (ex. ${exampleTag}). ${warnMsg}`
`Warning!\nCandidate tag: ${candidate} does not match the expected version ${release.version.version} (ex. ${exampleTag}). ${warnMsg}`
)
}
const owner = 'sourcegraph'
const repo = 'sourcegraph'
const releaseTag = `v${release.version}`
const releaseTag = `v${release.version.version}`
try {
const client = await getAuthenticatedGitHubClient()
@ -490,20 +490,23 @@ ${trackingIssues.map(index => `- ${slackURL(index.title, index.url)}`).join('\n'
argNames: ['version'],
run: async (config, version) => {
if (!version) {
const { upcoming: release } = await releaseVersions(config)
version = release.version
const release = await getActiveRelease(config)
version = release.version.version
}
const tags = getCandidateTags(localSourcegraphRepo, version)
console.log(`Release candidate tags for version: ${version}\n${tags}`)
console.log('To check the status of the build, run:\nsg ci status -branch tag\n')
if (tags.length > 0) {
console.log(`Release candidate tags for version: ${version}\n${tags}`)
console.log('To check the status of the build, run:\nsg ci status -branch tag\n')
} else {
console.log(chalk.yellow('No candidates found!'))
}
},
},
{
id: 'release:stage',
description: 'Open pull requests and a batch change staging a release',
run: async config => {
const { slackAnnounceChannel, dryRun } = config
const { upcoming: release, previous } = await releaseVersions(config)
const release = await getActiveRelease(config)
// ensure docker is running for 'batch changes'
try {
await ensureDocker()
@ -518,26 +521,28 @@ ${trackingIssues.map(index => `- ${slackURL(index.title, index.url)}`).join('\n'
await ensureSrcCliUpToDate()
// set up batch change config
const batchChange = batchChanges.releaseTrackingBatchChange(
release.version,
release.version.version,
await batchChanges.sourcegraphCLIConfig()
)
// default values
const notPatchRelease = release.patch === 0
const notPatchRelease = release.version.patch === 0
const versionRegex = '[0-9]+\\.[0-9]+\\.[0-9]+'
const batchChangeURL = batchChanges.batchChangeURL(batchChange)
const trackingIssue = await getTrackingIssue(await getAuthenticatedGitHubClient(), release)
const trackingIssue = await getTrackingIssue(await getAuthenticatedGitHubClient(), release.version)
if (!trackingIssue) {
throw new Error(`Tracking issue for version ${release.version} not found - has it been created yet?`)
throw new Error(
`Tracking issue for version ${release.version.version} not found - has it been created yet?`
)
}
// default PR content
const defaultPRMessage = `release: sourcegraph@${release.version}`
const defaultPRMessage = `release: sourcegraph@${release.version.version}`
const prBodyAndDraftState = (
actionItems: string[],
customMessage?: string
): { draft: boolean; body: string } => {
const defaultBody = `This pull request is part of the Sourcegraph ${release.version} release.
const defaultBody = `This pull request is part of the Sourcegraph ${release.version.version} release.
${customMessage || ''}
* [Release batch change](${batchChangeURL})
@ -560,7 +565,7 @@ These steps must be completed before this PR can be merged, unless otherwise sta
${actionItems.map(item => `- [ ] ${item}`).join('\n')}
cc @${config.captainGitHubUsername}
cc @${release.captainGitHubUsername}
`,
}
@ -574,36 +579,36 @@ cc @${config.captainGitHubUsername}
owner: 'sourcegraph',
repo: 'sourcegraph',
base: 'main',
head: `publish-${release.version}`,
head: `publish-${release.version.version}`,
commitMessage: notPatchRelease
? `draft sourcegraph@${release.version} release`
? `draft sourcegraph@${release.version.version} release`
: defaultPRMessage,
title: defaultPRMessage,
edits: [
// Update references to Sourcegraph versions in docs
`${sed} -i -E 's/version \`${versionRegex}\`/version \`${release.version}\`/g' doc/index.md`,
`${sed} -i -E 's/version \`${versionRegex}\`/version \`${release.version.version}\`/g' doc/index.md`,
// Update sourcegraph/server:VERSION everywhere except changelog
`find . -type f -name '*.md' ! -name 'CHANGELOG.md' -exec ${sed} -i -E 's/sourcegraph\\/server:${versionRegex}/sourcegraph\\/server:${release.version}/g' {} +`,
`find . -type f -name '*.md' ! -name 'CHANGELOG.md' -exec ${sed} -i -E 's/sourcegraph\\/server:${versionRegex}/sourcegraph\\/server:${release.version.version}/g' {} +`,
// Update Sourcegraph versions in installation guides
`find ./doc/admin/deploy/ -type f -name '*.md' -exec ${sed} -i -E 's/SOURCEGRAPH_VERSION="v${versionRegex}"/SOURCEGRAPH_VERSION="v${release.version}"/g' {} +`,
`find ./doc/admin/deploy/ -type f -name '*.md' -exec ${sed} -i -E 's/--version ${versionRegex}/--version ${release.version}/g' {} +`,
`find ./doc/admin/deploy/ -type f -name '*.md' -exec ${sed} -i -E 's/SOURCEGRAPH_VERSION="v${versionRegex}"/SOURCEGRAPH_VERSION="v${release.version.version}"/g' {} +`,
`find ./doc/admin/deploy/ -type f -name '*.md' -exec ${sed} -i -E 's/--version ${versionRegex}/--version ${release.version.version}/g' {} +`,
// Update fork variables in installation guides
`find ./doc/admin/deploy/ -type f -name '*.md' -exec ${sed} -i -E "s/DEPLOY_SOURCEGRAPH_DOCKER_FORK_REVISION='v${versionRegex}'/DEPLOY_SOURCEGRAPH_DOCKER_FORK_REVISION='v${release.version}'/g" {} +`,
`find ./doc/admin/deploy/ -type f -name '*.md' -exec ${sed} -i -E "s/DEPLOY_SOURCEGRAPH_DOCKER_FORK_REVISION='v${versionRegex}'/DEPLOY_SOURCEGRAPH_DOCKER_FORK_REVISION='v${release.version.version}'/g" {} +`,
notPatchRelease
? `comby -in-place '{{$previousReleaseRevspec := ":[1]"}} {{$previousReleaseVersion := ":[2]"}} {{$currentReleaseRevspec := ":[3]"}} {{$currentReleaseVersion := ":[4]"}}' '{{$previousReleaseRevspec := ":[3]"}} {{$previousReleaseVersion := ":[4]"}} {{$currentReleaseRevspec := "v${release.version}"}} {{$currentReleaseVersion := "${release.major}.${release.minor}"}}' doc/_resources/templates/document.html`
: `comby -in-place 'currentReleaseRevspec := ":[1]"' 'currentReleaseRevspec := "v${release.version}"' doc/_resources/templates/document.html`,
? `comby -in-place '{{$previousReleaseRevspec := ":[1]"}} {{$previousReleaseVersion := ":[2]"}} {{$currentReleaseRevspec := ":[3]"}} {{$currentReleaseVersion := ":[4]"}}' '{{$previousReleaseRevspec := ":[3]"}} {{$previousReleaseVersion := ":[4]"}} {{$currentReleaseRevspec := "v${release.version.version}"}} {{$currentReleaseVersion := "${release.version.major}.${release.version.minor}"}}' doc/_resources/templates/document.html`
: `comby -in-place 'currentReleaseRevspec := ":[1]"' 'currentReleaseRevspec := "v${release.version.version}"' doc/_resources/templates/document.html`,
// Update references to Sourcegraph deployment versions
`comby -in-place 'latestReleaseKubernetesBuild = newBuild(":[1]")' "latestReleaseKubernetesBuild = newBuild(\\"${release.version}\\")" cmd/frontend/internal/app/updatecheck/handler.go`,
`comby -in-place 'latestReleaseDockerServerImageBuild = newBuild(":[1]")' "latestReleaseDockerServerImageBuild = newBuild(\\"${release.version}\\")" cmd/frontend/internal/app/updatecheck/handler.go`,
`comby -in-place 'latestReleaseDockerComposeOrPureDocker = newBuild(":[1]")' "latestReleaseDockerComposeOrPureDocker = newBuild(\\"${release.version}\\")" cmd/frontend/internal/app/updatecheck/handler.go`,
`comby -in-place 'latestReleaseKubernetesBuild = newBuild(":[1]")' "latestReleaseKubernetesBuild = newBuild(\\"${release.version.version}\\")" cmd/frontend/internal/app/updatecheck/handler.go`,
`comby -in-place 'latestReleaseDockerServerImageBuild = newBuild(":[1]")' "latestReleaseDockerServerImageBuild = newBuild(\\"${release.version.version}\\")" cmd/frontend/internal/app/updatecheck/handler.go`,
`comby -in-place 'latestReleaseDockerComposeOrPureDocker = newBuild(":[1]")' "latestReleaseDockerComposeOrPureDocker = newBuild(\\"${release.version.version}\\")" cmd/frontend/internal/app/updatecheck/handler.go`,
// Support current release as the "previous release" going forward
notPatchRelease
? `comby -in-place 'const minimumUpgradeableVersion = ":[1]"' 'const minimumUpgradeableVersion = "${release.version}"' enterprise/dev/ci/internal/ci/*.go`
? `comby -in-place 'const minimumUpgradeableVersion = ":[1]"' 'const minimumUpgradeableVersion = "${release.version.version}"' enterprise/dev/ci/internal/ci/*.go`
: 'echo "Skipping minimumUpgradeableVersion bump on patch release"',
updateUpgradeGuides(previous.version, release.version),
updateUpgradeGuides(release.previous.version, release.version.version),
],
...prBodyAndDraftState(
((): string[] => {
@ -621,12 +626,12 @@ cc @${config.captainGitHubUsername}
owner: 'sourcegraph',
repo: 'about',
base: 'main',
head: `publish-${release.version}`,
head: `publish-${release.version.version}`,
commitMessage: defaultPRMessage,
title: defaultPRMessage,
edits: [
// Update sourcegraph/server:VERSION in all tsx files
`find . -type f -name '*.tsx' -exec ${sed} -i -E 's/sourcegraph\\/server:${versionRegex}/sourcegraph\\/server:${release.version}/g' {} +`,
`find . -type f -name '*.tsx' -exec ${sed} -i -E 's/sourcegraph\\/server:${versionRegex}/sourcegraph\\/server:${release.version.version}/g' {} +`,
],
...prBodyAndDraftState(
[],
@ -636,52 +641,52 @@ cc @${config.captainGitHubUsername}
{
owner: 'sourcegraph',
repo: 'deploy-sourcegraph',
base: `${release.major}.${release.minor}`,
head: `publish-${release.version}`,
base: release.branch,
head: `publish-${release.version.version}`,
commitMessage: defaultPRMessage,
title: defaultPRMessage,
edits: [`tools/update-docker-tags.sh ${release.version}`],
edits: [`tools/update-docker-tags.sh ${release.version.version}`],
...prBodyAndDraftState([]),
},
{
owner: 'sourcegraph',
repo: 'deploy-sourcegraph-k8s',
base: `${release.major}.${release.minor}`,
head: `publish-${release.version}`,
base: release.branch,
head: `publish-${release.version.version}`,
commitMessage: defaultPRMessage,
title: defaultPRMessage,
edits: [`sg ops update-images -pin-tag ${release.version} base/`],
edits: [`sg ops update-images -pin-tag ${release.version.version} base/`],
...prBodyAndDraftState([]),
},
{
owner: 'sourcegraph',
repo: 'deploy-sourcegraph-docker',
base: `${release.major}.${release.minor}`,
head: `publish-${release.version}`,
base: release.branch,
head: `publish-${release.version.version}`,
commitMessage: defaultPRMessage,
title: defaultPRMessage,
edits: [`tools/update-docker-tags.sh ${release.version}`],
edits: [`tools/update-docker-tags.sh ${release.version.version}`],
...prBodyAndDraftState([]),
},
{
owner: 'sourcegraph',
repo: 'deploy-sourcegraph-docker-customer-replica-1',
base: `${release.major}.${release.minor}`,
head: `publish-${release.version}`,
base: release.branch,
head: `publish-${release.version.version}`,
commitMessage: defaultPRMessage,
title: defaultPRMessage,
edits: [`tools/update-docker-tags.sh ${release.version}`],
edits: [`tools/update-docker-tags.sh ${release.version.version}`],
...prBodyAndDraftState([]),
},
{
owner: 'sourcegraph',
repo: 'deploy-sourcegraph-aws',
base: 'master',
head: `publish-${release.version}`,
head: `publish-${release.version.version}`,
commitMessage: defaultPRMessage,
title: defaultPRMessage,
edits: [
`${sed} -i -E 's/export SOURCEGRAPH_VERSION=${versionRegex}/export SOURCEGRAPH_VERSION=${release.version}/g' resources/amazon-linux2.sh`,
`${sed} -i -E 's/export SOURCEGRAPH_VERSION=${versionRegex}/export SOURCEGRAPH_VERSION=${release.version.version}/g' resources/amazon-linux2.sh`,
],
...prBodyAndDraftState([]),
},
@ -689,42 +694,42 @@ cc @${config.captainGitHubUsername}
owner: 'sourcegraph',
repo: 'deploy-sourcegraph-digitalocean',
base: 'master',
head: `publish-${release.version}`,
head: `publish-${release.version.version}`,
commitMessage: defaultPRMessage,
title: defaultPRMessage,
edits: [
`${sed} -i -E 's/export SOURCEGRAPH_VERSION=${versionRegex}/export SOURCEGRAPH_VERSION=${release.version}/g' resources/user-data.sh`,
`${sed} -i -E 's/export SOURCEGRAPH_VERSION=${versionRegex}/export SOURCEGRAPH_VERSION=${release.version.version}/g' resources/user-data.sh`,
],
...prBodyAndDraftState([]),
},
{
owner: 'sourcegraph',
repo: 'deploy-sourcegraph-helm',
base: `release/${release.major}.${release.minor}`,
head: `publish-${release.version}`,
base: `release/${release.branch}`,
head: `publish-${release.version.version}`,
commitMessage: defaultPRMessage,
title: defaultPRMessage,
edits: [
`for i in charts/*; do sg ops update-images -kind helm -pin-tag ${release.version} $i/.; done`,
`${sed} -i 's/appVersion:.*/appVersion: "${release.version}"/g' charts/*/Chart.yaml`,
`${sed} -i 's/version:.*/version: "${release.version}"/g' charts/*/Chart.yaml`,
`for i in charts/*; do sg ops update-images -kind helm -pin-tag ${release.version.version} $i/.; done`,
`${sed} -i 's/appVersion:.*/appVersion: "${release.version.version}"/g' charts/*/Chart.yaml`,
`${sed} -i 's/version:.*/version: "${release.version.version}"/g' charts/*/Chart.yaml`,
'./scripts/helm-docs.sh',
],
...prBodyAndDraftState([]),
},
],
dryRun: dryRun.changesets,
dryRun: config.dryRun.changesets,
})
// if changesets were actually published, set up a batch change and post in Slack
if (!dryRun.changesets) {
if (!config.dryRun.changesets) {
// Create batch change to track changes
try {
console.log(`Creating batch change in ${batchChange.cliConfig.SRC_ENDPOINT}`)
await batchChanges.createBatchChange(
createdChanges,
batchChange,
`Track publishing of sourcegraph v${release.version}: ${trackingIssue?.url}`
`Track publishing of sourcegraph v${release.version.version}: ${trackingIssue?.url}`
)
} catch (error) {
console.error(error)
@ -732,12 +737,12 @@ cc @${config.captainGitHubUsername}
}
// Announce release update in Slack
if (!dryRun.slack) {
if (!config.dryRun.slack) {
await postMessage(
`:captain: *Sourcegraph ${release.version} has been staged.*
`:captain: *Sourcegraph ${release.version.version} has been staged.*
Batch change: ${batchChangeURL}`,
slackAnnounceChannel
config.metadata.slackAnnounceChannel
)
}
}
@ -749,13 +754,13 @@ Batch change: ${batchChangeURL}`,
argNames: ['changeRepo', 'changeID'],
// Example: pnpm run release release:add-to-batch-change sourcegraph/about 1797
run: async (config, changeRepo, changeID) => {
const { upcoming: release } = await releaseVersions(config)
const release = await getActiveRelease(config)
if (!changeRepo || !changeID) {
throw new Error('Missing parameters (required: version, repo, change ID)')
}
const batchChange = batchChanges.releaseTrackingBatchChange(
release.version,
release.version.version,
await batchChanges.sourcegraphCLIConfig()
)
await batchChanges.addToBatchChange(
@ -774,13 +779,12 @@ Batch change: ${batchChangeURL}`,
id: 'release:finalize',
description: 'Run final tasks for sourcegraph/sourcegraph release pull requests',
run: async config => {
const { upcoming: release } = await releaseVersions(config)
const release = await getActiveRelease(config)
let failed = false
const owner = 'sourcegraph'
// Push final tags
const branch = `${release.major}.${release.minor}`
const tag = `v${release.version}`
const tag = `v${release.version.version}`
for (const repo of [
'deploy-sourcegraph',
'deploy-sourcegraph-docker',
@ -790,7 +794,7 @@ Batch change: ${batchChangeURL}`,
try {
const client = await getAuthenticatedGitHubClient()
const { workdir } = await cloneRepo(client, owner, repo, {
revision: branch,
revision: release.branch,
revisionMustExist: true,
})
await createTag(
@ -799,14 +803,14 @@ Batch change: ${batchChangeURL}`,
{
owner,
repo,
branch,
branch: release.branch,
tag,
},
config.dryRun.tags || false
)
} catch (error) {
console.error(error)
console.error(`Failed to create tag ${tag} on ${repo}@${branch}`)
console.error(`Failed to create tag ${tag} on ${repo}@${release.branch}`)
failed = true
}
}
@ -820,8 +824,7 @@ Batch change: ${batchChangeURL}`,
id: 'release:announce',
description: 'Announce a release as live',
run: async config => {
const { slackAnnounceChannel, dryRun } = config
const { upcoming: release } = await releaseVersions(config)
const release = await getActiveRelease(config)
const githubClient = await getAuthenticatedGitHubClient()
// Create final GitHub release
@ -832,9 +835,9 @@ Batch change: ${batchChangeURL}`,
{
owner: 'sourcegraph',
repo: 'sourcegraph',
release,
release: release.version,
},
dryRun.tags
config.dryRun.tags
)
} catch (error) {
console.error('Failed to generate GitHub release:', error)
@ -843,32 +846,37 @@ Batch change: ${batchChangeURL}`,
// Set up announcement message
const batchChangeURL = batchChanges.batchChangeURL(
batchChanges.releaseTrackingBatchChange(release.version, await batchChanges.sourcegraphCLIConfig())
batchChanges.releaseTrackingBatchChange(
release.version.version,
await batchChanges.sourcegraphCLIConfig()
)
)
const releaseMessage = `*Sourcegraph ${release.version} has been published*
const releaseMessage = `*Sourcegraph ${release.version.version} has been published*
* Changelog: ${changelogURL(release.format())}
* Changelog: ${changelogURL(release.version.format())}
* GitHub release: ${githubRelease || 'No release generated'}
* Release batch change: ${batchChangeURL}`
// Slack
const slackMessage = `:captain: ${releaseMessage}`
if (!dryRun.slack) {
await postMessage(slackMessage, slackAnnounceChannel)
console.log(`Posted to Slack channel ${slackAnnounceChannel}`)
if (!config.dryRun.slack) {
await postMessage(slackMessage, config.metadata.slackAnnounceChannel)
console.log(`Posted to Slack channel ${config.metadata.slackAnnounceChannel}`)
} else {
console.log(`dryRun enabled, skipping Slack post to ${slackAnnounceChannel}: ${slackMessage}`)
console.log(
`dryRun enabled, skipping Slack post to ${config.metadata.slackAnnounceChannel}: ${slackMessage}`
)
}
// GitHub tracking issues
const trackingIssue = await getTrackingIssue(githubClient, release)
const trackingIssue = await getTrackingIssue(githubClient, release.version)
if (!trackingIssue) {
console.warn(`Could not find tracking issue for release ${release.version} - skipping`)
console.warn(`Could not find tracking issue for release ${release.version.version} - skipping`)
} else {
// Note patch release requests if there are any outstanding
let comment = `${releaseMessage}
@${config.captainGitHubUsername}: Please complete the post-release steps before closing this issue.`
@${release.captainGitHubUsername}: Please complete the post-release steps before closing this issue.`
const patchRequestIssues = await queryIssues(githubClient, '', [IssueLabel.PATCH_REQUEST])
if (patchRequestIssues && patchRequestIssues.length > 0) {
comment += `
@ -876,7 +884,7 @@ Please also update outstanding patch requests, if relevant:
${patchRequestIssues.map(issue => `* #${issue.number}`).join('\n')}`
}
if (!dryRun.trackingIssues) {
if (!config.dryRun.trackingIssues) {
const commentURL = await commentOnIssue(githubClient, trackingIssue, comment)
console.log(`Please make sure to follow up on the release issue: ${commentURL}`)
} else {
@ -889,9 +897,48 @@ ${patchRequestIssues.map(issue => `* #${issue.number}`).join('\n')}`
id: 'release:close',
description: 'Close tracking issues for current release',
run: async config => {
const { previous: release } = await releaseVersions(config)
const active = await getActiveRelease(config)
// close tracking issue
await closeTrackingIssue(release)
await closeTrackingIssue(active.version)
console.log(chalk.blue('Deactivating release...'))
removeScheduledRelease(config, active.version.version)
deactivateAllReleases(config)
console.log(chalk.green(`Release ${active.version.format()} closed!`))
},
},
{
id: 'release:prepare',
description: 'Schedule a release',
run: async config => {
const rel = await newReleaseFromInput()
addScheduledRelease(config, rel)
saveReleaseConfig(config)
},
},
{
id: 'release:remove',
description: 'Remove a release from the config',
argNames: ['version'],
run: async (config, version) => {
await verifyWithInput(`Confirm you want to remove release: ${version} from the release config?`)
const rconfig = loadReleaseConfig()
removeScheduledRelease(rconfig, version)
saveReleaseConfig(rconfig)
},
},
{
id: 'release:activate-release',
description: 'Activate a feature release',
run: async config => {
await activateRelease(config)
},
},
{
id: 'release:deactivate-release',
description: 'Activate a feature release',
run: async config => {
await verifyWithInput('Are you sure you want to deactivate all releases?')
deactivateAllReleases(config)
},
},
{
@ -899,10 +946,10 @@ ${patchRequestIssues.map(issue => `* #${issue.number}`).join('\n')}`
description:
'Bake stitched migration files into the build for a release version. Only required for minor / major versions.',
run: async config => {
const { upcoming } = await releaseVersions(config)
const release = await getActiveRelease(config)
const releaseBranch = `${upcoming.major}.${upcoming.minor}`
const version = upcoming.version
const releaseBranch = release.branch
const version = release.version.version
ensureReleaseBranchUpToDate(releaseBranch)
const prConfig = {
@ -948,6 +995,22 @@ ${patchRequestIssues.map(issue => `* #${issue.number}`).join('\n')}`
rmdirSync(cacheFolder, { recursive: true })
},
},
{
id: 'util:previous-version',
description: 'Calculate the previous version based on repo tags',
argNames: ['version'],
run: (config: ReleaseConfig, version?: string) => {
let ver: SemVer | undefined
if (version) {
ver = new SemVer(version)
console.log(`Getting previous version from: ${version}...`)
} else {
console.log('Getting previous version...')
}
const prev = getPreviousVersion(ver)
console.log(chalk.green(`${prev.format()}`))
},
},
{
id: '_test:release-guide-content',
description: 'Generate upgrade guides',
@ -971,11 +1034,12 @@ ${patchRequestIssues.map(issue => `* #${issue.number}`).join('\n')}`
description: 'Test Google Calendar integration',
run: async config => {
const googleCalendar = await getClient()
const release = await getActiveRelease(config)
await ensureEvent(
{
title: 'TEST EVENT',
startDateTime: new Date(config.releaseDate).toISOString(),
endDateTime: addMinutes(new Date(config.releaseDate), 1).toISOString(),
startDateTime: new Date(release.releaseDate).toISOString(),
endDateTime: addMinutes(new Date(release.releaseDate), 1).toISOString(),
transparency: 'transparent',
},
googleCalendar
@ -1025,7 +1089,7 @@ ${patchRequestIssues.map(issue => `* #${issue.number}`).join('\n')}`
id: '_test:config',
description: 'Test release configuration loading',
run: config => {
console.log(JSON.stringify(config, null, ' '))
console.log(JSON.stringify(config, null, 2))
},
},
{

37
dev/release/src/util.ts
View File

@ -64,17 +64,12 @@ async function readLineNoCache(prompt: string): Promise<string> {
export async function verifyWithInput(prompt: string): Promise<void> {
await readLineNoCache(chalk.yellow(`${prompt}\nInput yes to confirm: `)).then(val => {
if (val !== 'yes') {
throw new Error()
console.log(chalk.red('Aborting!'))
process.exit(0)
}
})
}
export function getWeekNumber(date: Date): number {
const firstJan = new Date(date.getFullYear(), 0, 1)
const day = 86400000
return Math.ceil(((date.valueOf() - firstJan.valueOf()) / day + firstJan.getDay() + 1) / 7)
}
export async function ensureDocker(): Promise<execa.ExecaReturnValue<string>> {
return execa('docker', ['version'], { stdout: 'ignore' })
}
@ -206,13 +201,13 @@ export async function getContainerRegistryCredential(registryHostname: string):
return credential
}
export type ContentFunc = (previousVersion: string, nextVersion: string) => string
export type ContentFunc = (previousVersion?: string, nextVersion?: string) => string
const upgradeContentGenerators: { [s: string]: ContentFunc } = {
docker_compose: (previousVersion: string, nextVersion: string) => '',
kubernetes: (previousVersion: string, nextVersion: string) => '',
server: (previousVersion: string, nextVersion: string) => '',
pure_docker: (previousVersion: string, nextVersion: string) => {
docker_compose: () => '',
kubernetes: () => '',
server: () => '',
pure_docker: (previousVersion?: string, nextVersion?: string) => {
const compare = `compare/v${previousVersion}...v${nextVersion}`
return `As a template, perform the same actions as the following diff in your own deployment: [\`Upgrade to v${nextVersion}\`](https://github.com/sourcegraph/deploy-sourcegraph-docker/${compare})
\nFor non-standard replica builds:
@ -275,3 +270,21 @@ export const updateUpgradeGuides = (previous: string, next: string): EditFunc =>
}
}
}
export async function retryInput(
prompt: string,
delegate: (val: string) => boolean,
errorMessage?: string
): Promise<string> {
while (true) {
const val = await readLine(prompt).then(value => value)
if (delegate(val)) {
return val
}
if (errorMessage) {
console.log(chalk.red(errorMessage))
} else {
console.log(chalk.red('invalid input'))
}
}
}

19
dev/release/templates/patch_release_issue_template.md
View File

@ -27,7 +27,12 @@ This release is scheduled for **$RELEASE_DATE**.
<!-- Keep in sync with release_issue_template's "Setup" section -->
- [ ] Ensure release configuration in [`dev/release/release-config.jsonc`](https://sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/dev/release/release-config.jsonc) on `main` is up-to-date with the parameters for the current release.
- [ ] Ensure release configuration in [`dev/release/release-config.jsonc`](https://sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/dev/release/release-config.jsonc) on `main` has version $MAJOR.$MINOR.$PATCH selected by using the command:
```shell
pnpm run release release:activate-release
```
- [ ] Ensure you have the latest version of the release tooling and configuration by checking out and updating `sourcegraph@main`.
- [ ] Create a `Security release approval issue` and post a message in the [#security](https://sourcegraph.slack.com/archives/C1JH2BEHZ) channel tagging @security-support.
@ -109,14 +114,16 @@ Create and test the first release candidate:
- [ ] Finalize and announce that the release is live:
```sh
pnpm run release release:announce
pnpm run release release:close
```
## Post-release
- [ ] Open a PR to update [`dev/release/release-config.jsonc`](https://github.com/sourcegraph/sourcegraph/edit/main/dev/release/release-config.jsonc) with the parameters for the current release.
- [ ] Ensure you have the latest version of the release tooling and configuration by checking out and updating `sourcegraph@main`.
- [ ] Let the #cloud team know about the managed instances upgrade issue.
- [ ] Close this issue.
- [ ] Close the release:
```shell
pnpm run release release:close
```
- [ ] Open a PR to update [`dev/release/release-config.jsonc`](https://github.com/sourcegraph/sourcegraph/edit/main/dev/release/release-config.jsonc) after the auto-generated changes above.
**Note:** If another patch release is requested after the release, ask that a [patch request issue](https://github.com/sourcegraph/sourcegraph/issues/new?assignees=&labels=team%2Fdistribution&template=request_patch_release.md) be filled out and approved first.

30
dev/release/templates/release_issue_template.md
View File

@ -6,9 +6,8 @@ Arguments:
- $MINOR
- $PATCH
- $RELEASE_DATE
- $ONE_WORKING_WEEK_BEFORE_RELEASE
- $THREE_WORKING_DAY_BEFORE_RELEASE
- $ONE_WORKING_DAY_AFTER_RELEASE
- $SECURITY_REVIEW_DATE
- $CODE_FREEZE_DATE
-->
# $MAJOR.$MINOR release
@ -19,14 +18,19 @@ This release is scheduled for **$RELEASE_DATE**.
## Setup
- [ ] Ensure release configuration in [`dev/release/release-config.jsonc`](https://sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/dev/release/release-config.jsonc) on `main` is up to date with the parameters for the current release.
- [ ] Ensure release configuration in [`dev/release/release-config.jsonc`](https://sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/dev/release/release-config.jsonc) on `main` has version $MAJOR.$MINOR.$PATCH selected by using the command:
```shell
pnpm run release release:activate-release
```
- [ ] Ensure you have the latest version of the release tooling and configuration by checking out and updating `sourcegraph@main`.
## Security review (one week before release - $ONE_WORKING_WEEK_BEFORE_RELEASE)
## Security review (one week before release - $SECURITY_REVIEW_DATE)
- [ ] Create a [new issue](https://github.com/sourcegraph/sourcegraph/issues/new/choose) using the **Security release approval** template and post a message in the [#security](https://sourcegraph.slack.com/archives/C1JH2BEHZ) channel tagging `@security-support`.
## Cut release (three days before release - $THREE_WORKING_DAY_BEFORE_RELEASE)
## Cut release (three days before release - $CODE_FREEZE_DATE)
Perform these steps three days before the release date to generate a stable release candidate.
@ -134,29 +138,19 @@ On the day of the release, confirm there are no more release-blocking issues (as
- [ ] Cherry pick the release-publishing PR from the release branch into main
- [ ] Alert the marketing team in [#release-post](https://sourcegraph.slack.com/archives/C022Y5VUSBU) that they can merge the release post.
- [ ] Finalize and announce that the release is live:
```sh
pnpm run release release:announce
```
### Post-release
- [ ] Notify the next release captain that they are on duty for the next release.
- [ ] Open a PR to update [`dev/release/release-config.jsonc`](https://sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/dev/release/release-config.jsonc) with the parameters for the next release.
- [ ] Change `upcomingRelease` to the current patch release
- [ ] Change `previousRelease` to the previous patch release version
- [ ] Change `releaseDate` to the current date (time is optional) along with `oneWorkingDayAfterRelease` and `threeWorkingDaysBeforeRelease`
- [ ] Change `captainSlackUsername` and `captainGitHubUsername` accordingly
- [ ] Ensure you have the latest version of the release tooling and configuration by checking out and updating `sourcegraph@main`.
- [ ] Create release calendar events, tracking issue, and announcement for next release:
- [ ] Create release calendar events, tracking issue, and announcement for next release (note: these commands will prompt for user input to generate the definition for the next release):
```sh
pnpm run release tracking:issues
pnpm run release tracking:timeline
```
- [ ] Open a PR to update [`dev/release/release-config.jsonc`](https://sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/dev/release/release-config.jsonc) with the auto-generated changes from above.
- [ ] Close the release.
```sh
pnpm run release release:close
```

7
pnpm-lock.yaml
View File

@ -1182,7 +1182,12 @@ importers:
'@sourcegraph/testing': link:../testing
dev/release:
specifiers: {}
specifiers:
'@types/luxon': ^3.2.0
luxon: ^3.2.1
dependencies:
'@types/luxon': 3.2.0
luxon: 3.2.1
schema:
specifiers: {}