From dd1e512434655210cb62b231dadf76efecc53a50 Mon Sep 17 00:00:00 2001 From: coury-clark Date: Fri, 17 Feb 2023 11:00:42 -0700 Subject: [PATCH] release-tool: refactor release config (#47711) Closes [#47557](https://github.com/sourcegraph/sourcegraph/issues/47557) This is a somewhat gruesome PR to read, so sorry for that. I wanted to keep this change atomic so it's easy to revert if we need. So, I set out to automate the dates in the release config and realized that fundamentally the architecture of the release tool was coupled to this config in a way that was making life more difficult than it needed to be. This PR refactors the release config in the release tool. It uses an entirely new schema, and for the most part automates every interaction anyone will have with the config. The new schema is designed to do a few things: 1. Deprecate the annoying cached version by specifying two sections of the release config, both a definition and an active releases field. Anything in the active releases is considered in progress. 2. Sets up some architecture to support multi-release functionality from the release tool. There are guard rails right now since this isn't fully supported. Also, * Automatically detect and suggest versions * Automatically generate release dates from a specified date or current time * QOL changes to flow to the appropriate commands if the release config is in the wrong state. For example it will prompt for input if the config is empty for a given version for commands that require a valid state * Guardrails around every interaction I could find that causes problems * New commands to interact with the release config (activate-release, deactivate-release, util:previous-version) ## Usage To set a release as active, either use the activate command directly ``` pnpm run release release:activate-release ``` or transitively through another dependent command. All release commands that interact with the _current_ release will read from the active release in the release config. All release commands that interact with _future_ releases will read and write to the release definitions in the release config. To deactivate the release: ``` pnpm run release release:deactivate-release ``` Most of these steps are not explicitly mandatory, and are coupled into other relevant commands (for example closing the release will deactivate the release also). To add new scheduled release definitions: ``` pnpm run release release:prepare ``` To remove scheduled releases: ``` pnpm run release release:remove ``` To determine previous versions: ``` pnpm run release util:previous-version 4.2.1 Getting previous version from: 4.2.1... 4.2.0 ``` or ``` pnpm run release util:previous-version Getting previous version... 4.4.2 ``` ## Test plan I did a lot of manual testing since so many commands were impacted. Some example output is below, and here is an example of a fake issue that was generated from the release tool: https://github.com/sourcegraph/sourcegraph/issues/47760 Example of how the input will flow if the config is not in the correct state, including version suggestions: ``` > @sourcegraph/dev-release@0.0.1 release /Users/coury@sourcegraph.com/Documents/code/sourcegraph/dev/release > ts-node --transpile-only ./src/main.ts "release:status" No active releases are defined! Attempting to activate... Next minor release: 4.5.0 Next patch release: 4.4.3 Enter the version to activate: ``` ``` > @sourcegraph/dev-release@0.0.1 release /Users/coury@sourcegraph.com/Documents/code/sourcegraph/dev/release > ts-node --transpile-only ./src/main.ts "release:status" No active releases are defined! Attempting to activate... Next minor release: 4.5.0 Next patch release: 4.4.3 Enter the version to activate: 4.4.3 Attempting to detect previous version... Detected previous version: 4.4.2 Release definition not found for: 4.4.3, enter release information. Enter the release date (YYYY-MM-DD). Enter blank to use current date: Using current time: 2023-02-16T11:01:34.710-08:00 Enter the github username of the release captain: coury-clark Enter the slack username of the release captain: coury Version created: { "codeFreezeDate": "2023-02-09T10:01:34.710-08:00", "securityApprovalDate": "2023-02-09T10:01:34.710-08:00", "releaseDate": "2023-02-16T10:01:34.710-08:00", "current": "4.4.3", "captainGitHubUsername": "coury-clark", "captainSlackUsername": "coury" } Release: 4.4.3 activated! ``` ## App preview: - [Web](https://sg-web-cclark-refactor-release-config.onrender.com/search) Check out the [client app preview documentation](https://docs.sourcegraph.com/dev/how-to/client_pr_previews) to learn more. --- dev/release/package.json | 4 + dev/release/release-config.jsonc | 54 +-- dev/release/src/config.ts | 260 ++++++++-- dev/release/src/git.ts | 48 ++ dev/release/src/github.ts | 63 +-- dev/release/src/main.ts | 4 +- dev/release/src/release.ts | 450 ++++++++++-------- dev/release/src/util.ts | 37 +- .../templates/patch_release_issue_template.md | 19 +- .../templates/release_issue_template.md | 30 +- pnpm-lock.yaml | 7 +- 11 files changed, 610 insertions(+), 366 deletions(-) create mode 100644 dev/release/src/git.ts diff --git a/dev/release/package.json b/dev/release/package.json index 0aa0bb9dfb1..1fedb76e847 100644 --- a/dev/release/package.json +++ b/dev/release/package.json @@ -6,5 +6,9 @@ "scripts": { "release": "ts-node --transpile-only ./src/main.ts", "lint:js": "eslint --cache 'src/**/*.[jt]s?(x)'" + }, + "dependencies": { + "@types/luxon": "^3.2.0", + "luxon": "^3.2.1" } } diff --git a/dev/release/release-config.jsonc b/dev/release/release-config.jsonc index e7b62315c8a..51fd9c9d761 100644 --- a/dev/release/release-config.jsonc +++ b/dev/release/release-config.jsonc @@ -1,39 +1,23 @@ -/** - * Configuration for the release tool. All fields should align exactly with those in - * `src/config.ts`. To learn more about the release tool: - * - * pnpm run release help - * - * A quick sanity-check can be performed to make sure configuration is loaded correctly: - * - * pnpm run release _test:config - */ { - // Captain information - // To get your slack username, navigate to Profile -> More -> Account settings -> Username (at the bottom) -> Expand - "captainSlackUsername": "keegan", - "captainGitHubUsername": "keegancsmith", - // Release versions - "previousRelease": "4.4.2", - "upcomingRelease": "4.5.0", - "oneWorkingWeekBeforeRelease": "15 February 2023 10:00 PST", - "threeWorkingDaysBeforeRelease": "17 February 2023 10:00 PST", - "releaseDate": "22 February 2023 10:00 PST", - "oneWorkingDayAfterRelease": "23 February 2023 10:00 PST", - "oneWorkingWeekAfterRelease": "1 March 2023 10:00 PST", - // Channel where messages from the tooling are posted - "slackAnnounceChannel": "announce-engineering", - // Email for preparing calendar events + "metadata": { "teamEmail": "team@sourcegraph.com", - // Enable dry-running for certain features - useful for testing or sanity-checking. - // - // For example, `dryRun.changesets` prints changes generated by `release:stage` - // instead of pushing them to GitHub. - "dryRun": { - "tags": false, - "changesets": false, - "trackingIssues": false, - "calendar": false, - "slack": false + "slackAnnounceChannel": "announce-engineering" + }, + "scheduledReleases": { + "4.5.0": { + "codeFreezeDate": "2023-02-15T10:00:00.000-08:00", + "securityApprovalDate": "2023-02-15T10:00:00.000-08:00", + "releaseDate": "2023-02-22T10:00:00.000-08:00", + "current": "4.5.0", + "captainGitHubUsername": "coury-clark", + "captainSlackUsername": "coury" } + }, + "dryRun": { + "tags": false, + "changesets": false, + "trackingIssues": false, + "calendar": false, + "slack": false + } } diff --git a/dev/release/src/config.ts b/dev/release/src/config.ts index 72fecec6869..9754ac823b7 100644 --- a/dev/release/src/config.ts +++ b/dev/release/src/config.ts @@ -1,9 +1,15 @@ -import { readFileSync, unlinkSync } from 'fs' +import { readFileSync, writeFileSync } from 'fs' +import chalk from 'chalk' import { parse as parseJSONC } from 'jsonc-parser' +import { DateTime } from 'luxon' import * as semver from 'semver' +import { SemVer } from 'semver' -import { cacheFolder, readLine, getWeekNumber } from './util' +import { getPreviousVersion } from './git' +import { retryInput } from './util' + +const releaseConfigPath = 'release-config.jsonc' /** * Release configuration file format @@ -34,63 +40,213 @@ export interface Config { } } -/** - * Default path of JSONC containing release configuration. - */ -const configPath = 'release-config.jsonc' - -/** - * Loads configuration from predefined path. It does not do any special validation. - */ -export function loadConfig(): Config { - return parseJSONC(readFileSync(configPath).toString()) as Config +export async function getActiveRelease(config: ReleaseConfig): Promise { + if (!config.in_progress || config.in_progress.releases.length === 0) { + console.log(chalk.yellow('No active releases are defined! Attempting to activate...')) + await activateRelease(config) + } + if (!config.in_progress) { + throw new Error('unable to activate a release!') + } + if (config.in_progress.releases.length > 1) { + throw new Error( + chalk.red( + 'The release config has multiple versions activated. This feature is not yet supported by the release tool! Please activate only a single release.' + ) + ) + } + const rel = config.in_progress.releases[0] + const def = config.scheduledReleases[rel.version] + const version = new SemVer(rel.version) + return { + version, + previous: new SemVer(rel.previous), + ...(def as ReleaseDates), + ...(def as ReleaseCaptainInformation), + branch: `${version.minor}.${version.minor}`, + } } -/** - * Convenience function for getting relevant configured releases as semver.SemVer - * - * It prompts for a confirmation of the `upcomingRelease` that is cached for a week. - */ -export async function releaseVersions(config: Config): Promise<{ - previous: semver.SemVer - upcoming: semver.SemVer -}> { - const parseOptions: semver.Options = { loose: false } - const parsedPrevious = semver.parse(config.previousRelease, parseOptions) - if (!parsedPrevious) { - throw new Error(`config.previousRelease '${config.previousRelease}' is not valid semver`) +export function loadReleaseConfig(): ReleaseConfig { + return parseJSONC(readFileSync(releaseConfigPath).toString()) as ReleaseConfig +} + +export function saveReleaseConfig(config: ReleaseConfig): void { + writeFileSync(releaseConfigPath, JSON.stringify(config, null, 2)) +} + +export function newRelease( + version: SemVer, + releaseDate: DateTime, + captainGithub: string, + captainSlack: string +): ScheduledReleaseDefinition { + return { + ...releaseDates(releaseDate), + current: version.version, + captainGitHubUsername: captainGithub, + captainSlackUsername: captainSlack, } - const parsedUpcoming = semver.parse(config.upcomingRelease, parseOptions) - if (!parsedUpcoming) { - throw new Error(`config.upcomingRelease '${config.upcomingRelease}' is not valid semver`) +} + +export async function newReleaseFromInput(versionOverride?: SemVer): Promise { + let version = versionOverride + if (!version) { + version = await selectVersionWithSuggestion('Enter the desired version number') } - // Verify the configured upcoming release. The response is cached and expires in a - // week, after which the captain is required to confirm again. - const now = new Date() - const cachedVersionResponse = `${cacheFolder}/current_release_${now.getUTCFullYear()}_${getWeekNumber(now)}.txt` - const confirmVersion = await readLine( - `Please confirm the upcoming release version configured in '${configPath}' (currently '${config.upcomingRelease}') by entering it again: `, - cachedVersionResponse + const releaseDateStr = await retryInput( + 'Enter the release date (YYYY-MM-DD). Enter blank to use current date: ', + val => { + if (val && /^\d{4}-\d{2}-\d{2}$/.test(val)) { + return true + } + // this will return false if the input doesn't match the regexp above but does exist, allowing blank input to still be valid + return !val + }, + 'invalid date, expected format YYYY-MM-DD' ) - const parsedConfirmed = semver.parse(confirmVersion, parseOptions) - let error = '' - if (!parsedConfirmed) { - error = `Provided version '${confirmVersion}' is not valid semver` - } else if (semver.neq(parsedConfirmed, parsedUpcoming)) { - error = `Provided version '${confirmVersion}' and config.upcomingRelease '${config.upcomingRelease}' do not match - please update the release configuration at '${configPath}' and try again` + let releaseTime: DateTime + if (!releaseDateStr) { + releaseTime = DateTime.now().setZone('America/Los_Angeles') + console.log(chalk.blue(`Using current time: ${releaseTime.toString()}`)) + } else { + releaseTime = DateTime.fromISO(releaseDateStr, { zone: 'America/Los_Angeles' }) } - // If error, abort and remove the cached response (since it is invalid anyway) - if (error !== '') { - unlinkSync(cachedVersionResponse) - throw new Error(error) - } + const captainGithubUsername = await retryInput('Enter the github username of the release captain: ', val => !!val) + const captainSlackUsername = await retryInput('Enter the slack username of the release captain: ', val => !!val) - const versions = { - previous: parsedPrevious, - upcoming: parsedUpcoming, - } - console.log(`Using versions: { upcoming: ${versions.upcoming.format()}, previous: ${versions.previous.format()} }`) - return versions + const rel = newRelease(version, releaseTime, captainGithubUsername, captainSlackUsername) + console.log(chalk.green('Version created:')) + console.log(chalk.green(JSON.stringify(rel, null, 2))) + return rel +} + +function releaseDates(releaseDate: DateTime): ReleaseDates { + releaseDate = releaseDate.set({ hour: 10 }) + return { + codeFreezeDate: releaseDate.plus({ days: -7 }).toString(), + securityApprovalDate: releaseDate.plus({ days: -7 }).toString(), + releaseDate: releaseDate.toString(), + } +} + +export function addScheduledRelease(config: ReleaseConfig, release: ScheduledReleaseDefinition): ReleaseConfig { + config.scheduledReleases[release.current] = release + return config +} + +export function removeScheduledRelease(config: ReleaseConfig, version: string): ReleaseConfig { + delete config.scheduledReleases[version] + return config +} + +export interface ReleaseDates { + releaseDate: string + codeFreezeDate: string + securityApprovalDate: string +} + +export interface ActiveRelease extends ReleaseCaptainInformation, ReleaseDates { + version: SemVer + previous: SemVer + branch: string +} + +export interface ActiveReleaseDefinition { + version: string + previous: string +} + +export interface ReleaseCaptainInformation { + captainSlackUsername: string + captainGitHubUsername: string +} + +export interface InProgress extends ReleaseCaptainInformation { + releases: ActiveReleaseDefinition[] +} + +export interface ReleaseConfig { + metadata: { + teamEmail: string + slackAnnounceChannel: string + } + scheduledReleases: { + [version: string]: ScheduledReleaseDefinition + } + in_progress?: InProgress + dryRun: { + tags?: boolean + changesets?: boolean + trackingIssues?: boolean + slack?: boolean + calendar?: boolean + } +} + +export interface ScheduledReleaseDefinition extends ReleaseDates, ReleaseCaptainInformation { + current: string +} + +// Prompt a user for input and activate the given release version if possible. Will redirect to release creation input if +// the version isn't defined. +export async function activateRelease(config: ReleaseConfig): Promise { + const next = await selectVersionWithSuggestion('Enter the version to activate') + console.log('Attempting to detect previous version...') + const previous = getPreviousVersion(next) + console.log(chalk.blue(`Detected previous version: ${previous.version}`)) + + const scheduled = await getScheduledReleaseWithInput(config, next) + config.in_progress = { + captainGitHubUsername: scheduled.captainGitHubUsername, + captainSlackUsername: scheduled.captainSlackUsername, + releases: [{ version: next.version, previous: previous.version }], + } + saveReleaseConfig(config) + console.log(chalk.green(`Release: ${next.version} activated!`)) +} + +export function deactivateAllReleases(config: ReleaseConfig): void { + delete config.in_progress + saveReleaseConfig(config) +} + +// Prompt a user for a major / minor version input with automation suggestion by adding a minor version to the previous version. +async function selectVersionWithSuggestion(prompt: string): Promise { + const probablyMinor = getPreviousVersion().inc('minor') + const probablyPatch = getPreviousVersion().inc('patch') + const input = await retryInput( + `Next minor release: ${probablyMinor.version}\nNext patch release: ${probablyPatch.version}\n${chalk.blue( + prompt + )}: `, + val => { + const version = semver.parse(val) + return !!version + } + ) + return new SemVer(input) +} + +// Prompt a user for a release definition input, and redirect to creation input if it doesn't exist. +export async function getReleaseDefinition(config: ReleaseConfig): Promise { + const next = await selectVersionWithSuggestion('Enter the version number to select') + return getScheduledReleaseWithInput(config, next) +} + +// Helper function to get a release definition from the release config, redirecting to creation input if it doesn't exist. +async function getScheduledReleaseWithInput( + config: ReleaseConfig, + releaseVersion: SemVer +): Promise { + let scheduled = config.scheduledReleases[releaseVersion.version] + if (!scheduled) { + console.log( + chalk.yellow(`Release definition not found for: ${releaseVersion.version}, enter release information.\n`) + ) + scheduled = await newReleaseFromInput(releaseVersion) + addScheduledRelease(config, scheduled) + } + return scheduled } diff --git a/dev/release/src/git.ts b/dev/release/src/git.ts new file mode 100644 index 00000000000..9e649fdf708 --- /dev/null +++ b/dev/release/src/git.ts @@ -0,0 +1,48 @@ +import execa from 'execa' +import { SemVer } from 'semver' +import * as semver from 'semver' + +import { localSourcegraphRepo } from './github' + +export function getTags(workdir: string, prefix?: string): string[] { + execa.sync('git', ['fetch', '--tags'], { cwd: workdir }) + return execa + .sync('git', ['--no-pager', 'tag', '-l', `${prefix}`, '--sort=v:refname'], { cwd: workdir }) + .stdout.split('\n') +} + +export function getCandidateTags(workdir: string, version: string): string[] { + return getTags(workdir, `v${version}-rc*`) +} + +export function getReleaseTags(workdir: string): string[] { + const raw = getTags(workdir, 'v[0-9]*.[0-9]*.[0-9]*') + // since tags are globbed they can overmatch when we just want pure release tags + return raw.filter(tag => tag.match('[0-9]+\\.[0-9]+\\.[0-9]+$')) +} + +// Returns the version tagged in the repository previous to a provided input version. If no input version it will +// simply return the highest version found in the repository. +export function getPreviousVersion(version?: SemVer): SemVer { + const lowest = new SemVer('0.0.1') + const tags = getReleaseTags(localSourcegraphRepo) + if (tags.length === 0) { + return lowest + } + if (!version) { + return new SemVer(tags[tags.length - 1]) + } + + for ( + let reallyLongVariableNameBecauseESLintRulesAreSilly = tags.length - 1; + reallyLongVariableNameBecauseESLintRulesAreSilly >= 0; + reallyLongVariableNameBecauseESLintRulesAreSilly-- + ) { + const tag = tags[reallyLongVariableNameBecauseESLintRulesAreSilly] + const temp = semver.parse(tag) + if (temp && temp.compare(version) === -1) { + return temp + } + } + return lowest +} diff --git a/dev/release/src/github.ts b/dev/release/src/github.ts index 5dc8675cd4f..1d7be3608dc 100644 --- a/dev/release/src/github.ts +++ b/dev/release/src/github.ts @@ -1,4 +1,4 @@ -import { mkdtemp as original_mkdtemp, readFileSync, existsSync } from 'fs' +import { existsSync, mkdtemp as original_mkdtemp, readFileSync } from 'fs' import * as os from 'os' import * as path from 'path' import { promisify } from 'util' @@ -9,7 +9,8 @@ import execa from 'execa' import fetch from 'node-fetch' import * as semver from 'semver' -import { readLine, formatDate, timezoneLink, cacheFolder, changelogURL, getContainerRegistryCredential } from './util' +import { cacheFolder, changelogURL, formatDate, getContainerRegistryCredential, readLine, timezoneLink } from './util' + const mkdtemp = promisify(original_mkdtemp) let githubPAT: string @@ -88,21 +89,17 @@ interface IssueTemplateArguments { */ version: semver.SemVer /** - * Available as `$ONE_WORKING_WEEK_BEFORE_RELEASE` + * Available as `$SECURITY_REVIEW_DATE` */ - oneWorkingWeekBeforeRelease: Date + securityReviewDate: Date /** - * Available as `$ONE_WORKING_DAY_BEFORE_RELEASE` + * Available as `$CODE_FREEZE_DATE` */ - threeWorkingDaysBeforeRelease: Date + codeFreezeDate: Date /** * Available as `$RELEASE_DATE` */ releaseDate: Date - /** - * Available as `$ONE_WORKING_DAY_AFTER_RELEASE` - */ - oneWorkingDayAfterRelease: Date } /** @@ -143,13 +140,7 @@ function dateMarkdown(date: Date, name: string): string { async function execTemplate( octokit: Octokit, template: IssueTemplate, - { - version, - oneWorkingWeekBeforeRelease, - threeWorkingDaysBeforeRelease, - releaseDate, - oneWorkingDayAfterRelease, - }: IssueTemplateArguments + { version, securityReviewDate, codeFreezeDate, releaseDate }: IssueTemplateArguments ): Promise { console.log(`Preparing issue from ${JSON.stringify(template)}`) const name = releaseName(version) @@ -158,19 +149,9 @@ async function execTemplate( .replace(/\$MAJOR/g, version.major.toString()) .replace(/\$MINOR/g, version.minor.toString()) .replace(/\$PATCH/g, version.patch.toString()) - .replace( - /\$ONE_WORKING_WEEK_BEFORE_RELEASE/g, - dateMarkdown(oneWorkingWeekBeforeRelease, `One working week before ${name} release`) - ) - .replace( - /\$THREE_WORKING_DAY_BEFORE_RELEASE/g, - dateMarkdown(threeWorkingDaysBeforeRelease, `Three working days before ${name} release`) - ) + .replace(/\$SECURITY_REVIEW_DATE/g, dateMarkdown(securityReviewDate, `One working week before ${name} release`)) + .replace(/\$CODE_FREEZE_DATE/g, dateMarkdown(codeFreezeDate, `Three working days before ${name} release`)) .replace(/\$RELEASE_DATE/g, dateMarkdown(releaseDate, `${name} release date`)) - .replace( - /\$ONE_WORKING_DAY_AFTER_RELEASE/g, - dateMarkdown(oneWorkingDayAfterRelease, `One working day after ${name} release`) - ) } interface MaybeIssue { @@ -189,17 +170,15 @@ export async function ensureTrackingIssues({ version, assignees, releaseDate, - oneWorkingWeekBeforeRelease, - threeWorkingDaysBeforeRelease, - oneWorkingDayAfterRelease, + securityReviewDate, + codeFreezeDate, dryRun, }: { version: semver.SemVer assignees: string[] releaseDate: Date - oneWorkingWeekBeforeRelease: Date - threeWorkingDaysBeforeRelease: Date - oneWorkingDayAfterRelease: Date + securityReviewDate: Date + codeFreezeDate: Date dryRun: boolean }): Promise { const octokit = await getAuthenticatedGitHubClient() @@ -234,9 +213,8 @@ export async function ensureTrackingIssues({ const body = await execTemplate(octokit, template, { version, releaseDate, - oneWorkingWeekBeforeRelease, - threeWorkingDaysBeforeRelease, - oneWorkingDayAfterRelease, + securityReviewDate, + codeFreezeDate, }) const issue = await ensureIssue( octokit, @@ -735,12 +713,3 @@ export async function closeTrackingIssue(version: semver.SemVer): Promise await closeIssue(octokit, previousIssue) } } - -export function getTags(workdir: string, prefix: string): string[] { - execa.sync('git', ['fetch', '--tags'], { cwd: workdir }) - return execa.sync('git', ['--no-pager', 'tag', '-l', `${prefix}`], { cwd: workdir }).stdout.split('\t') -} - -export function getCandidateTags(workdir: string, version: string): string[] { - return getTags(workdir, `v${version}-rc*`) -} diff --git a/dev/release/src/main.ts b/dev/release/src/main.ts index 772bc9787b7..dfa15b8f1d1 100644 --- a/dev/release/src/main.ts +++ b/dev/release/src/main.ts @@ -1,4 +1,4 @@ -import { loadConfig } from './config' +import { loadReleaseConfig } from './config' import { runStep, StepID } from './release' import { ensureMainBranchUpToDate } from './util' @@ -6,7 +6,7 @@ import { ensureMainBranchUpToDate } from './util' * Release captain automation */ async function main(): Promise { - const config = loadConfig() + const config = loadReleaseConfig() const args = process.argv.slice(2) if (args.length === 0) { await runStep(config, 'help') diff --git a/dev/release/src/release.ts b/dev/release/src/release.ts index ed2881a34e6..f31cf93e711 100644 --- a/dev/release/src/release.ts +++ b/dev/release/src/release.ts @@ -1,44 +1,57 @@ import { readFileSync, rmdirSync, writeFileSync } from 'fs' import * as path from 'path' +import chalk from 'chalk' import commandExists from 'command-exists' import { addMinutes } from 'date-fns' import execa from 'execa' +import { SemVer } from 'semver' import * as batchChanges from './batchChanges' import * as changelog from './changelog' -import { Config, releaseVersions } from './config' import { - getAuthenticatedGitHubClient, - listIssues, - getTrackingIssue, + activateRelease, + addScheduledRelease, + loadReleaseConfig, + newReleaseFromInput, + ReleaseConfig, + getActiveRelease, + removeScheduledRelease, + saveReleaseConfig, + getReleaseDefinition, + deactivateAllReleases, +} from './config' +import { getCandidateTags, getPreviousVersion } from './git' +import { + cloneRepo, + closeTrackingIssue, + commentOnIssue, createChangesets, CreatedChangeset, + createLatestRelease, createTag, ensureTrackingIssues, - closeTrackingIssue, - releaseName, - commentOnIssue, - queryIssues, + getAuthenticatedGitHubClient, + getTrackingIssue, IssueLabel, - createLatestRelease, - cloneRepo, - getCandidateTags, + listIssues, localSourcegraphRepo, + queryIssues, + releaseName, } from './github' -import { ensureEvent, getClient, EventOptions, calendarTime } from './google-calendar' +import { calendarTime, ensureEvent, EventOptions, getClient } from './google-calendar' import { postMessage, slackURL } from './slack' import { cacheFolder, - formatDate, - timezoneLink, - ensureDocker, changelogURL, + ensureDocker, ensureReleaseBranchUpToDate, ensureSrcCliEndpoint, ensureSrcCliUpToDate, - getLatestTag, + formatDate, getAllUpgradeGuides, + getLatestTag, + timezoneLink, updateUpgradeGuides, verifyWithInput, } from './util' @@ -64,8 +77,13 @@ export type StepID = | 'release:announce' | 'release:close' | 'release:multi-version-bake' + | 'release:prepare' + | 'release:remove' + | 'release:activate-release' + | 'release:deactivate-release' // util | 'util:clear-cache' + | 'util:previous-version' // testing | '_test:google-calendar' | '_test:slack' @@ -79,7 +97,7 @@ export type StepID = /** * Runs given release step with the provided configuration and arguments. */ -export async function runStep(config: Config, step: StepID, ...args: string[]): Promise { +export async function runStep(config: ReleaseConfig, step: StepID, ...args: string[]): Promise { if (!steps.map(({ id }) => id as string).includes(step)) { throw new Error(`Unrecognized step ${JSON.stringify(step)}`) } @@ -97,7 +115,9 @@ export async function runStep(config: Config, step: StepID, ...args: string[]): interface Step { id: StepID description: string - run?: ((config: Config, ...args: string[]) => Promise) | ((config: Config, ...args: string[]) => void) + run?: + | ((config: ReleaseConfig, ...args: string[]) => Promise) + | ((config: ReleaseConfig, ...args: string[]) => void) argNames?: string[] } @@ -132,48 +152,32 @@ const steps: Step[] = [ id: 'tracking:timeline', description: 'Generate a set of Google Calendar events for a MAJOR.MINOR release', run: async config => { - const { upcoming: release } = await releaseVersions(config) - const name = releaseName(release) + const next = await getReleaseDefinition(config) + const name = releaseName(new SemVer(next.current)) const events: EventOptions[] = [ { title: `Security Team to Review Release Container Image Scans ${name}`, description: '(This is not an actual event to attend, just a calendar marker.)', anyoneCanAddSelf: true, - attendees: [config.teamEmail], + attendees: [config.metadata.teamEmail], transparency: 'transparent', - ...calendarTime(config.oneWorkingWeekBeforeRelease), + ...calendarTime(next.securityApprovalDate), }, { title: `Cut Sourcegraph ${name}`, description: '(This is not an actual event to attend, just a calendar marker.)', anyoneCanAddSelf: true, - attendees: [config.teamEmail], + attendees: [config.metadata.teamEmail], transparency: 'transparent', - ...calendarTime(config.threeWorkingDaysBeforeRelease), + ...calendarTime(next.codeFreezeDate), }, { title: `Release Sourcegraph ${name}`, description: '(This is not an actual event to attend, just a calendar marker.)', anyoneCanAddSelf: true, - attendees: [config.teamEmail], + attendees: [config.metadata.teamEmail], transparency: 'transparent', - ...calendarTime(config.releaseDate), - }, - { - title: `Start deploying Sourcegraph ${name} to Cloud instances`, - description: '(This is not an actual event to attend, just a calendar marker.)', - anyoneCanAddSelf: true, - attendees: [config.teamEmail], - transparency: 'transparent', - ...calendarTime(config.oneWorkingDayAfterRelease), - }, - { - title: `All Cloud instances upgraded to Sourcegraph ${name}`, - description: '(This is not an actual event to attend, just a calendar marker.)', - anyoneCanAddSelf: true, - attendees: [config.teamEmail], - transparency: 'transparent', - ...calendarTime(config.oneWorkingWeekAfterRelease), + ...calendarTime(next.releaseDate), }, ] @@ -190,53 +194,43 @@ const steps: Step[] = [ }, { id: 'tracking:issues', - description: 'Generate GitHub tracking issue for the configured release', - run: async (config: Config) => { - const { - releaseDate, - captainGitHubUsername, - oneWorkingWeekBeforeRelease, - threeWorkingDaysBeforeRelease, - oneWorkingDayAfterRelease, - captainSlackUsername, - slackAnnounceChannel, - dryRun, - } = config - const { upcoming: release } = await releaseVersions(config) - const date = new Date(releaseDate) + description: 'Generate GitHub tracking issues for a release', + run: async (config: ReleaseConfig) => { + const next = await getReleaseDefinition(config) + const version = new SemVer(next.current) + const date = new Date(next.releaseDate) // Create issue const trackingIssues = await ensureTrackingIssues({ - version: release, - assignees: [captainGitHubUsername], + version, + assignees: [next.captainGitHubUsername], releaseDate: date, - oneWorkingWeekBeforeRelease: new Date(oneWorkingWeekBeforeRelease), - threeWorkingDaysBeforeRelease: new Date(threeWorkingDaysBeforeRelease), - oneWorkingDayAfterRelease: new Date(oneWorkingDayAfterRelease), - dryRun: dryRun.trackingIssues || false, + securityReviewDate: new Date(next.securityApprovalDate), + codeFreezeDate: new Date(next.codeFreezeDate), + dryRun: config.dryRun.trackingIssues || false, }) console.log('Rendered tracking issues', trackingIssues) // If at least one issue was created, post to Slack if (trackingIssues.find(({ created }) => created)) { - const name = releaseName(release) + const name = releaseName(version) const releaseDateString = slackURL(formatDate(date), timezoneLink(date, `${name} release`)) let annoncement = `:mega: *${name} release* -:captain: Release captain: @${captainSlackUsername} +:captain: Release captain: @${next.captainSlackUsername} :spiral_calendar_pad: Scheduled for: ${releaseDateString} :pencil: Tracking issues: ${trackingIssues.map(index => `- ${slackURL(index.title, index.url)}`).join('\n')}` - if (release.patch !== 0) { - const patchRequestTemplate = `https://github.com/sourcegraph/sourcegraph/issues/new?assignees=&labels=team%2Fdistribution&template=request_patch_release.md&title=${release.version}%3A+` + if (version.patch !== 0) { + const patchRequestTemplate = `https://github.com/sourcegraph/sourcegraph/issues/new?assignees=&labels=team%2Fdistribution&template=request_patch_release.md&title=${version.version}%3A+` annoncement += `\n\nIf you have changes that should go into this patch release, ${slackURL( 'please *file a patch request issue*', patchRequestTemplate )}, or it will not be included.` } - if (!dryRun.slack) { - await postMessage(annoncement, slackAnnounceChannel) - console.log(`Posted to Slack channel ${slackAnnounceChannel}`) + if (!config.dryRun.slack) { + await postMessage(annoncement, config.metadata.slackAnnounceChannel) + console.log(`Posted to Slack channel ${config.metadata.slackAnnounceChannel}`) } } else { console.log('No tracking issues were created, skipping Slack announcement') @@ -248,8 +242,8 @@ ${trackingIssues.map(index => `- ${slackURL(index.title, index.url)}`).join('\n' description: 'Generate pull requests to perform a changelog cut for branch cut', argNames: ['changelogFile'], run: async (config, changelogFile = 'CHANGELOG.md') => { - const { upcoming: release } = await releaseVersions(config) - const prMessage = `changelog: cut sourcegraph@${release.version}` + const upcoming = await getActiveRelease(config) + const prMessage = `changelog: cut sourcegraph@${upcoming.version.version}` const pullRequest = await createChangesets({ requiredCommands: [], changes: [ @@ -257,17 +251,17 @@ ${trackingIssues.map(index => `- ${slackURL(index.title, index.url)}`).join('\n' owner: 'sourcegraph', repo: 'sourcegraph', base: 'main', - head: `changelog-${release.version}`, + head: `changelog-${upcoming.version.version}`, title: prMessage, commitMessage: prMessage + '\n\n ## Test plan\n\nn/a', edits: [ (directory: string) => { - console.log(`Updating '${changelogFile} for ${release.format()}'`) + console.log(`Updating '${changelogFile} for ${upcoming.version.format()}'`) const changelogPath = path.join(directory, changelogFile) let changelogContents = readFileSync(changelogPath).toString() // Convert 'unreleased' to a release - const releaseHeader = `## ${release.format()}` + const releaseHeader = `## ${upcoming.version.format()}` const unreleasedHeader = '## Unreleased' changelogContents = changelogContents.replace(unreleasedHeader, releaseHeader) @@ -286,20 +280,20 @@ ${trackingIssues.map(index => `- ${slackURL(index.title, index.url)}`).join('\n' owner: 'sourcegraph', repo: 'deploy-sourcegraph-helm', base: 'main', - head: `changelog-${release.version}`, + head: `changelog-${upcoming.version.version}`, title: prMessage, commitMessage: prMessage, body: prMessage + '\n\n ## Test plan\n\nn/a', edits: [ (directory: string) => { - console.log(`Updating '${changelogFile} for ${release.format()}'`) + console.log(`Updating '${changelogFile} for ${upcoming.version.format()}'`) const changelogPath = path.join(directory, 'charts', 'sourcegraph', changelogFile) let changelogContents = readFileSync(changelogPath).toString() // Convert 'unreleased' to a release - const releaseHeader = `## ${release.format()}` + const releaseHeader = `## ${upcoming.version.format()}` const releaseUpdate = - releaseHeader + `\n\n- Sourcegraph ${release.format()} is now available\n` + releaseHeader + `\n\n- Sourcegraph ${upcoming.version.format()} is now available\n` const unreleasedHeader = '## Unreleased\n' changelogContents = changelogContents.replace(unreleasedHeader, releaseUpdate) @@ -327,21 +321,22 @@ ${trackingIssues.map(index => `- ${slackURL(index.title, index.url)}`).join('\n' id: 'release:branch-cut', description: 'Create release branch', run: async config => { - const { upcoming: release } = await releaseVersions(config) - const branch = `${release.major}.${release.minor}` + const release = await getActiveRelease(config) let message: string // notify cs team on patch release cut - if (release.patch !== 0) { - message = `:mega: *${release.version}* branch has been cut cc: @cs` + if (release.version.patch !== 0) { + message = `:mega: *${release.version.version}* branch has been cut cc: @cs` } else { - message = `:mega: *${release.version}* branch has been cut.` + message = `:mega: *${release.version.version}* branch has been cut.` } try { // Create and push new release branch from changelog commit - await execa('git', ['branch', branch]) - await execa('git', ['push', 'origin', branch]) - await postMessage(message, config.slackAnnounceChannel) - console.log(`To check the status of the branch, run:\nsg ci status -branch ${release.version} --wait\n`) + await execa('git', ['branch', release.branch]) + await execa('git', ['push', 'origin', release.branch]) + await postMessage(message, config.metadata.slackAnnounceChannel) + console.log( + `To check the status of the branch, run:\nsg ci status -branch ${release.version.version} --wait\n` + ) } catch (error) { console.error('Failed to create release branch', error) } @@ -352,11 +347,13 @@ ${trackingIssues.map(index => `- ${slackURL(index.title, index.url)}`).join('\n' description: 'Post a message in Slack summarizing the progress of a release', run: async config => { const githubClient = await getAuthenticatedGitHubClient() - const { upcoming: release } = await releaseVersions(config) + const release = await getActiveRelease(config) - const trackingIssue = await getTrackingIssue(githubClient, release) + const trackingIssue = await getTrackingIssue(githubClient, release.version) if (!trackingIssue) { - throw new Error(`Tracking issue for version ${release.version} not found - has it been created yet?`) + throw new Error( + `Tracking issue for version ${release.version.version} not found - has it been created yet?` + ) } const latestTag = (await getLatestTag('sourcegraph', 'sourcegraph')).toString() const latestBuildURL = `https://buildkite.com/sourcegraph/sourcegraph/builds?branch=${latestTag}` @@ -373,13 +370,15 @@ ${trackingIssues.map(index => `- ${slackURL(index.title, index.url)}`).join('\n' : `are ${blockingIssues.length} release-blocking issues` }` - const message = `:mega: *${release.version} Release Status Update* + const message = `:mega: *${release.version.version} Release Status Update* * Tracking issue: ${trackingIssue.url} * ${blockingMessage}: ${blockingIssuesURL} * ${latestBuildMessage}` if (!config.dryRun.slack) { - await postMessage(message, config.slackAnnounceChannel) + await postMessage(message, config.metadata.slackAnnounceChannel) + } else { + console.log(chalk.green('Dry run: ' + message)) } }, }, @@ -387,18 +386,20 @@ ${trackingIssues.map(index => `- ${slackURL(index.title, index.url)}`).join('\n' id: 'release:create-candidate', description: 'Generate the Nth release candidate. Set to "final" to generate a final release', run: async config => { - const { upcoming: release } = await releaseVersions(config) - const branch = `${release.major}.${release.minor}` - ensureReleaseBranchUpToDate(branch) + const release = await getActiveRelease(config) + ensureReleaseBranchUpToDate(release.branch) const owner = 'sourcegraph' const repo = 'sourcegraph' try { const client = await getAuthenticatedGitHubClient() - const { workdir } = await cloneRepo(client, owner, repo, { revision: branch, revisionMustExist: true }) + const { workdir } = await cloneRepo(client, owner, repo, { + revision: release.branch, + revisionMustExist: true, + }) - const tags = getCandidateTags(workdir, release.version) + const tags = getCandidateTags(workdir, release.version.version) let nextCandidate = 1 for (const tag of tags) { const num = parseInt(tag.slice(-1), 10) @@ -406,7 +407,7 @@ ${trackingIssues.map(index => `- ${slackURL(index.title, index.url)}`).join('\n' nextCandidate = num + 1 } } - const tag = `v${release.version}-rc.${nextCandidate}` + const tag = `v${release.version.version}-rc.${nextCandidate}` console.log(`Detected next candidate: ${nextCandidate}, attempting to create tag: ${tag}`) await createTag( @@ -415,7 +416,7 @@ ${trackingIssues.map(index => `- ${slackURL(index.title, index.url)}`).join('\n' { owner, repo, - branch, + branch: release.branch, tag, }, config.dryRun.tags || false @@ -432,13 +433,12 @@ ${trackingIssues.map(index => `- ${slackURL(index.title, index.url)}`).join('\n' 'Promote a release candidate to release build. Specify the full candidate tag to promote the tagged commit to release.', argNames: ['candidate'], run: async (config, candidate) => { - const { upcoming: release } = await releaseVersions(config) - const releaseBranch = `${release.major}.${release.minor}` - ensureReleaseBranchUpToDate(releaseBranch) + const release = await getActiveRelease(config) + ensureReleaseBranchUpToDate(release.branch) const warnMsg = 'Verify the provided tag is correct to promote to release. Note: it is very unusual to require a non-standard tag to promote to release, proceed with caution.' - const exampleTag = `v${release.version}-rc.1` + const exampleTag = `v${release.version.version}-rc.1` if (!candidate) { throw new Error( `Candidate tag is a required argument. This should be the git tag of the commit to promote to release (ex.${exampleTag}` @@ -447,16 +447,16 @@ ${trackingIssues.map(index => `- ${slackURL(index.title, index.url)}`).join('\n' await verifyWithInput( `Warning!\nCandidate tag: ${candidate} does not match the standard convention (ex. ${exampleTag}). ${warnMsg}` ) - } else if (!candidate.match(`${release.version}-rc\\.\\d`)) { + } else if (!candidate.match(`${release.version.version}-rc\\.\\d`)) { await verifyWithInput( - `Warning!\nCandidate tag: ${candidate} does not match the expected version ${release.version} (ex. ${exampleTag}). ${warnMsg}` + `Warning!\nCandidate tag: ${candidate} does not match the expected version ${release.version.version} (ex. ${exampleTag}). ${warnMsg}` ) } const owner = 'sourcegraph' const repo = 'sourcegraph' - const releaseTag = `v${release.version}` + const releaseTag = `v${release.version.version}` try { const client = await getAuthenticatedGitHubClient() @@ -490,20 +490,23 @@ ${trackingIssues.map(index => `- ${slackURL(index.title, index.url)}`).join('\n' argNames: ['version'], run: async (config, version) => { if (!version) { - const { upcoming: release } = await releaseVersions(config) - version = release.version + const release = await getActiveRelease(config) + version = release.version.version } const tags = getCandidateTags(localSourcegraphRepo, version) - console.log(`Release candidate tags for version: ${version}\n${tags}`) - console.log('To check the status of the build, run:\nsg ci status -branch tag\n') + if (tags.length > 0) { + console.log(`Release candidate tags for version: ${version}\n${tags}`) + console.log('To check the status of the build, run:\nsg ci status -branch tag\n') + } else { + console.log(chalk.yellow('No candidates found!')) + } }, }, { id: 'release:stage', description: 'Open pull requests and a batch change staging a release', run: async config => { - const { slackAnnounceChannel, dryRun } = config - const { upcoming: release, previous } = await releaseVersions(config) + const release = await getActiveRelease(config) // ensure docker is running for 'batch changes' try { await ensureDocker() @@ -518,26 +521,28 @@ ${trackingIssues.map(index => `- ${slackURL(index.title, index.url)}`).join('\n' await ensureSrcCliUpToDate() // set up batch change config const batchChange = batchChanges.releaseTrackingBatchChange( - release.version, + release.version.version, await batchChanges.sourcegraphCLIConfig() ) // default values - const notPatchRelease = release.patch === 0 + const notPatchRelease = release.version.patch === 0 const versionRegex = '[0-9]+\\.[0-9]+\\.[0-9]+' const batchChangeURL = batchChanges.batchChangeURL(batchChange) - const trackingIssue = await getTrackingIssue(await getAuthenticatedGitHubClient(), release) + const trackingIssue = await getTrackingIssue(await getAuthenticatedGitHubClient(), release.version) if (!trackingIssue) { - throw new Error(`Tracking issue for version ${release.version} not found - has it been created yet?`) + throw new Error( + `Tracking issue for version ${release.version.version} not found - has it been created yet?` + ) } // default PR content - const defaultPRMessage = `release: sourcegraph@${release.version}` + const defaultPRMessage = `release: sourcegraph@${release.version.version}` const prBodyAndDraftState = ( actionItems: string[], customMessage?: string ): { draft: boolean; body: string } => { - const defaultBody = `This pull request is part of the Sourcegraph ${release.version} release. + const defaultBody = `This pull request is part of the Sourcegraph ${release.version.version} release. ${customMessage || ''} * [Release batch change](${batchChangeURL}) @@ -560,7 +565,7 @@ These steps must be completed before this PR can be merged, unless otherwise sta ${actionItems.map(item => `- [ ] ${item}`).join('\n')} -cc @${config.captainGitHubUsername} +cc @${release.captainGitHubUsername} `, } @@ -574,36 +579,36 @@ cc @${config.captainGitHubUsername} owner: 'sourcegraph', repo: 'sourcegraph', base: 'main', - head: `publish-${release.version}`, + head: `publish-${release.version.version}`, commitMessage: notPatchRelease - ? `draft sourcegraph@${release.version} release` + ? `draft sourcegraph@${release.version.version} release` : defaultPRMessage, title: defaultPRMessage, edits: [ // Update references to Sourcegraph versions in docs - `${sed} -i -E 's/version \`${versionRegex}\`/version \`${release.version}\`/g' doc/index.md`, + `${sed} -i -E 's/version \`${versionRegex}\`/version \`${release.version.version}\`/g' doc/index.md`, // Update sourcegraph/server:VERSION everywhere except changelog - `find . -type f -name '*.md' ! -name 'CHANGELOG.md' -exec ${sed} -i -E 's/sourcegraph\\/server:${versionRegex}/sourcegraph\\/server:${release.version}/g' {} +`, + `find . -type f -name '*.md' ! -name 'CHANGELOG.md' -exec ${sed} -i -E 's/sourcegraph\\/server:${versionRegex}/sourcegraph\\/server:${release.version.version}/g' {} +`, // Update Sourcegraph versions in installation guides - `find ./doc/admin/deploy/ -type f -name '*.md' -exec ${sed} -i -E 's/SOURCEGRAPH_VERSION="v${versionRegex}"/SOURCEGRAPH_VERSION="v${release.version}"/g' {} +`, - `find ./doc/admin/deploy/ -type f -name '*.md' -exec ${sed} -i -E 's/--version ${versionRegex}/--version ${release.version}/g' {} +`, + `find ./doc/admin/deploy/ -type f -name '*.md' -exec ${sed} -i -E 's/SOURCEGRAPH_VERSION="v${versionRegex}"/SOURCEGRAPH_VERSION="v${release.version.version}"/g' {} +`, + `find ./doc/admin/deploy/ -type f -name '*.md' -exec ${sed} -i -E 's/--version ${versionRegex}/--version ${release.version.version}/g' {} +`, // Update fork variables in installation guides - `find ./doc/admin/deploy/ -type f -name '*.md' -exec ${sed} -i -E "s/DEPLOY_SOURCEGRAPH_DOCKER_FORK_REVISION='v${versionRegex}'/DEPLOY_SOURCEGRAPH_DOCKER_FORK_REVISION='v${release.version}'/g" {} +`, + `find ./doc/admin/deploy/ -type f -name '*.md' -exec ${sed} -i -E "s/DEPLOY_SOURCEGRAPH_DOCKER_FORK_REVISION='v${versionRegex}'/DEPLOY_SOURCEGRAPH_DOCKER_FORK_REVISION='v${release.version.version}'/g" {} +`, notPatchRelease - ? `comby -in-place '{{$previousReleaseRevspec := ":[1]"}} {{$previousReleaseVersion := ":[2]"}} {{$currentReleaseRevspec := ":[3]"}} {{$currentReleaseVersion := ":[4]"}}' '{{$previousReleaseRevspec := ":[3]"}} {{$previousReleaseVersion := ":[4]"}} {{$currentReleaseRevspec := "v${release.version}"}} {{$currentReleaseVersion := "${release.major}.${release.minor}"}}' doc/_resources/templates/document.html` - : `comby -in-place 'currentReleaseRevspec := ":[1]"' 'currentReleaseRevspec := "v${release.version}"' doc/_resources/templates/document.html`, + ? `comby -in-place '{{$previousReleaseRevspec := ":[1]"}} {{$previousReleaseVersion := ":[2]"}} {{$currentReleaseRevspec := ":[3]"}} {{$currentReleaseVersion := ":[4]"}}' '{{$previousReleaseRevspec := ":[3]"}} {{$previousReleaseVersion := ":[4]"}} {{$currentReleaseRevspec := "v${release.version.version}"}} {{$currentReleaseVersion := "${release.version.major}.${release.version.minor}"}}' doc/_resources/templates/document.html` + : `comby -in-place 'currentReleaseRevspec := ":[1]"' 'currentReleaseRevspec := "v${release.version.version}"' doc/_resources/templates/document.html`, // Update references to Sourcegraph deployment versions - `comby -in-place 'latestReleaseKubernetesBuild = newBuild(":[1]")' "latestReleaseKubernetesBuild = newBuild(\\"${release.version}\\")" cmd/frontend/internal/app/updatecheck/handler.go`, - `comby -in-place 'latestReleaseDockerServerImageBuild = newBuild(":[1]")' "latestReleaseDockerServerImageBuild = newBuild(\\"${release.version}\\")" cmd/frontend/internal/app/updatecheck/handler.go`, - `comby -in-place 'latestReleaseDockerComposeOrPureDocker = newBuild(":[1]")' "latestReleaseDockerComposeOrPureDocker = newBuild(\\"${release.version}\\")" cmd/frontend/internal/app/updatecheck/handler.go`, + `comby -in-place 'latestReleaseKubernetesBuild = newBuild(":[1]")' "latestReleaseKubernetesBuild = newBuild(\\"${release.version.version}\\")" cmd/frontend/internal/app/updatecheck/handler.go`, + `comby -in-place 'latestReleaseDockerServerImageBuild = newBuild(":[1]")' "latestReleaseDockerServerImageBuild = newBuild(\\"${release.version.version}\\")" cmd/frontend/internal/app/updatecheck/handler.go`, + `comby -in-place 'latestReleaseDockerComposeOrPureDocker = newBuild(":[1]")' "latestReleaseDockerComposeOrPureDocker = newBuild(\\"${release.version.version}\\")" cmd/frontend/internal/app/updatecheck/handler.go`, // Support current release as the "previous release" going forward notPatchRelease - ? `comby -in-place 'const minimumUpgradeableVersion = ":[1]"' 'const minimumUpgradeableVersion = "${release.version}"' enterprise/dev/ci/internal/ci/*.go` + ? `comby -in-place 'const minimumUpgradeableVersion = ":[1]"' 'const minimumUpgradeableVersion = "${release.version.version}"' enterprise/dev/ci/internal/ci/*.go` : 'echo "Skipping minimumUpgradeableVersion bump on patch release"', - updateUpgradeGuides(previous.version, release.version), + updateUpgradeGuides(release.previous.version, release.version.version), ], ...prBodyAndDraftState( ((): string[] => { @@ -621,12 +626,12 @@ cc @${config.captainGitHubUsername} owner: 'sourcegraph', repo: 'about', base: 'main', - head: `publish-${release.version}`, + head: `publish-${release.version.version}`, commitMessage: defaultPRMessage, title: defaultPRMessage, edits: [ // Update sourcegraph/server:VERSION in all tsx files - `find . -type f -name '*.tsx' -exec ${sed} -i -E 's/sourcegraph\\/server:${versionRegex}/sourcegraph\\/server:${release.version}/g' {} +`, + `find . -type f -name '*.tsx' -exec ${sed} -i -E 's/sourcegraph\\/server:${versionRegex}/sourcegraph\\/server:${release.version.version}/g' {} +`, ], ...prBodyAndDraftState( [], @@ -636,52 +641,52 @@ cc @${config.captainGitHubUsername} { owner: 'sourcegraph', repo: 'deploy-sourcegraph', - base: `${release.major}.${release.minor}`, - head: `publish-${release.version}`, + base: release.branch, + head: `publish-${release.version.version}`, commitMessage: defaultPRMessage, title: defaultPRMessage, - edits: [`tools/update-docker-tags.sh ${release.version}`], + edits: [`tools/update-docker-tags.sh ${release.version.version}`], ...prBodyAndDraftState([]), }, { owner: 'sourcegraph', repo: 'deploy-sourcegraph-k8s', - base: `${release.major}.${release.minor}`, - head: `publish-${release.version}`, + base: release.branch, + head: `publish-${release.version.version}`, commitMessage: defaultPRMessage, title: defaultPRMessage, - edits: [`sg ops update-images -pin-tag ${release.version} base/`], + edits: [`sg ops update-images -pin-tag ${release.version.version} base/`], ...prBodyAndDraftState([]), }, { owner: 'sourcegraph', repo: 'deploy-sourcegraph-docker', - base: `${release.major}.${release.minor}`, - head: `publish-${release.version}`, + base: release.branch, + head: `publish-${release.version.version}`, commitMessage: defaultPRMessage, title: defaultPRMessage, - edits: [`tools/update-docker-tags.sh ${release.version}`], + edits: [`tools/update-docker-tags.sh ${release.version.version}`], ...prBodyAndDraftState([]), }, { owner: 'sourcegraph', repo: 'deploy-sourcegraph-docker-customer-replica-1', - base: `${release.major}.${release.minor}`, - head: `publish-${release.version}`, + base: release.branch, + head: `publish-${release.version.version}`, commitMessage: defaultPRMessage, title: defaultPRMessage, - edits: [`tools/update-docker-tags.sh ${release.version}`], + edits: [`tools/update-docker-tags.sh ${release.version.version}`], ...prBodyAndDraftState([]), }, { owner: 'sourcegraph', repo: 'deploy-sourcegraph-aws', base: 'master', - head: `publish-${release.version}`, + head: `publish-${release.version.version}`, commitMessage: defaultPRMessage, title: defaultPRMessage, edits: [ - `${sed} -i -E 's/export SOURCEGRAPH_VERSION=${versionRegex}/export SOURCEGRAPH_VERSION=${release.version}/g' resources/amazon-linux2.sh`, + `${sed} -i -E 's/export SOURCEGRAPH_VERSION=${versionRegex}/export SOURCEGRAPH_VERSION=${release.version.version}/g' resources/amazon-linux2.sh`, ], ...prBodyAndDraftState([]), }, @@ -689,42 +694,42 @@ cc @${config.captainGitHubUsername} owner: 'sourcegraph', repo: 'deploy-sourcegraph-digitalocean', base: 'master', - head: `publish-${release.version}`, + head: `publish-${release.version.version}`, commitMessage: defaultPRMessage, title: defaultPRMessage, edits: [ - `${sed} -i -E 's/export SOURCEGRAPH_VERSION=${versionRegex}/export SOURCEGRAPH_VERSION=${release.version}/g' resources/user-data.sh`, + `${sed} -i -E 's/export SOURCEGRAPH_VERSION=${versionRegex}/export SOURCEGRAPH_VERSION=${release.version.version}/g' resources/user-data.sh`, ], ...prBodyAndDraftState([]), }, { owner: 'sourcegraph', repo: 'deploy-sourcegraph-helm', - base: `release/${release.major}.${release.minor}`, - head: `publish-${release.version}`, + base: `release/${release.branch}`, + head: `publish-${release.version.version}`, commitMessage: defaultPRMessage, title: defaultPRMessage, edits: [ - `for i in charts/*; do sg ops update-images -kind helm -pin-tag ${release.version} $i/.; done`, - `${sed} -i 's/appVersion:.*/appVersion: "${release.version}"/g' charts/*/Chart.yaml`, - `${sed} -i 's/version:.*/version: "${release.version}"/g' charts/*/Chart.yaml`, + `for i in charts/*; do sg ops update-images -kind helm -pin-tag ${release.version.version} $i/.; done`, + `${sed} -i 's/appVersion:.*/appVersion: "${release.version.version}"/g' charts/*/Chart.yaml`, + `${sed} -i 's/version:.*/version: "${release.version.version}"/g' charts/*/Chart.yaml`, './scripts/helm-docs.sh', ], ...prBodyAndDraftState([]), }, ], - dryRun: dryRun.changesets, + dryRun: config.dryRun.changesets, }) // if changesets were actually published, set up a batch change and post in Slack - if (!dryRun.changesets) { + if (!config.dryRun.changesets) { // Create batch change to track changes try { console.log(`Creating batch change in ${batchChange.cliConfig.SRC_ENDPOINT}`) await batchChanges.createBatchChange( createdChanges, batchChange, - `Track publishing of sourcegraph v${release.version}: ${trackingIssue?.url}` + `Track publishing of sourcegraph v${release.version.version}: ${trackingIssue?.url}` ) } catch (error) { console.error(error) @@ -732,12 +737,12 @@ cc @${config.captainGitHubUsername} } // Announce release update in Slack - if (!dryRun.slack) { + if (!config.dryRun.slack) { await postMessage( - `:captain: *Sourcegraph ${release.version} has been staged.* + `:captain: *Sourcegraph ${release.version.version} has been staged.* Batch change: ${batchChangeURL}`, - slackAnnounceChannel + config.metadata.slackAnnounceChannel ) } } @@ -749,13 +754,13 @@ Batch change: ${batchChangeURL}`, argNames: ['changeRepo', 'changeID'], // Example: pnpm run release release:add-to-batch-change sourcegraph/about 1797 run: async (config, changeRepo, changeID) => { - const { upcoming: release } = await releaseVersions(config) + const release = await getActiveRelease(config) if (!changeRepo || !changeID) { throw new Error('Missing parameters (required: version, repo, change ID)') } const batchChange = batchChanges.releaseTrackingBatchChange( - release.version, + release.version.version, await batchChanges.sourcegraphCLIConfig() ) await batchChanges.addToBatchChange( @@ -774,13 +779,12 @@ Batch change: ${batchChangeURL}`, id: 'release:finalize', description: 'Run final tasks for sourcegraph/sourcegraph release pull requests', run: async config => { - const { upcoming: release } = await releaseVersions(config) + const release = await getActiveRelease(config) let failed = false const owner = 'sourcegraph' // Push final tags - const branch = `${release.major}.${release.minor}` - const tag = `v${release.version}` + const tag = `v${release.version.version}` for (const repo of [ 'deploy-sourcegraph', 'deploy-sourcegraph-docker', @@ -790,7 +794,7 @@ Batch change: ${batchChangeURL}`, try { const client = await getAuthenticatedGitHubClient() const { workdir } = await cloneRepo(client, owner, repo, { - revision: branch, + revision: release.branch, revisionMustExist: true, }) await createTag( @@ -799,14 +803,14 @@ Batch change: ${batchChangeURL}`, { owner, repo, - branch, + branch: release.branch, tag, }, config.dryRun.tags || false ) } catch (error) { console.error(error) - console.error(`Failed to create tag ${tag} on ${repo}@${branch}`) + console.error(`Failed to create tag ${tag} on ${repo}@${release.branch}`) failed = true } } @@ -820,8 +824,7 @@ Batch change: ${batchChangeURL}`, id: 'release:announce', description: 'Announce a release as live', run: async config => { - const { slackAnnounceChannel, dryRun } = config - const { upcoming: release } = await releaseVersions(config) + const release = await getActiveRelease(config) const githubClient = await getAuthenticatedGitHubClient() // Create final GitHub release @@ -832,9 +835,9 @@ Batch change: ${batchChangeURL}`, { owner: 'sourcegraph', repo: 'sourcegraph', - release, + release: release.version, }, - dryRun.tags + config.dryRun.tags ) } catch (error) { console.error('Failed to generate GitHub release:', error) @@ -843,32 +846,37 @@ Batch change: ${batchChangeURL}`, // Set up announcement message const batchChangeURL = batchChanges.batchChangeURL( - batchChanges.releaseTrackingBatchChange(release.version, await batchChanges.sourcegraphCLIConfig()) + batchChanges.releaseTrackingBatchChange( + release.version.version, + await batchChanges.sourcegraphCLIConfig() + ) ) - const releaseMessage = `*Sourcegraph ${release.version} has been published* + const releaseMessage = `*Sourcegraph ${release.version.version} has been published* -* Changelog: ${changelogURL(release.format())} +* Changelog: ${changelogURL(release.version.format())} * GitHub release: ${githubRelease || 'No release generated'} * Release batch change: ${batchChangeURL}` // Slack const slackMessage = `:captain: ${releaseMessage}` - if (!dryRun.slack) { - await postMessage(slackMessage, slackAnnounceChannel) - console.log(`Posted to Slack channel ${slackAnnounceChannel}`) + if (!config.dryRun.slack) { + await postMessage(slackMessage, config.metadata.slackAnnounceChannel) + console.log(`Posted to Slack channel ${config.metadata.slackAnnounceChannel}`) } else { - console.log(`dryRun enabled, skipping Slack post to ${slackAnnounceChannel}: ${slackMessage}`) + console.log( + `dryRun enabled, skipping Slack post to ${config.metadata.slackAnnounceChannel}: ${slackMessage}` + ) } // GitHub tracking issues - const trackingIssue = await getTrackingIssue(githubClient, release) + const trackingIssue = await getTrackingIssue(githubClient, release.version) if (!trackingIssue) { - console.warn(`Could not find tracking issue for release ${release.version} - skipping`) + console.warn(`Could not find tracking issue for release ${release.version.version} - skipping`) } else { // Note patch release requests if there are any outstanding let comment = `${releaseMessage} -@${config.captainGitHubUsername}: Please complete the post-release steps before closing this issue.` +@${release.captainGitHubUsername}: Please complete the post-release steps before closing this issue.` const patchRequestIssues = await queryIssues(githubClient, '', [IssueLabel.PATCH_REQUEST]) if (patchRequestIssues && patchRequestIssues.length > 0) { comment += ` @@ -876,7 +884,7 @@ Please also update outstanding patch requests, if relevant: ${patchRequestIssues.map(issue => `* #${issue.number}`).join('\n')}` } - if (!dryRun.trackingIssues) { + if (!config.dryRun.trackingIssues) { const commentURL = await commentOnIssue(githubClient, trackingIssue, comment) console.log(`Please make sure to follow up on the release issue: ${commentURL}`) } else { @@ -889,9 +897,48 @@ ${patchRequestIssues.map(issue => `* #${issue.number}`).join('\n')}` id: 'release:close', description: 'Close tracking issues for current release', run: async config => { - const { previous: release } = await releaseVersions(config) + const active = await getActiveRelease(config) // close tracking issue - await closeTrackingIssue(release) + await closeTrackingIssue(active.version) + console.log(chalk.blue('Deactivating release...')) + removeScheduledRelease(config, active.version.version) + deactivateAllReleases(config) + console.log(chalk.green(`Release ${active.version.format()} closed!`)) + }, + }, + { + id: 'release:prepare', + description: 'Schedule a release', + run: async config => { + const rel = await newReleaseFromInput() + addScheduledRelease(config, rel) + saveReleaseConfig(config) + }, + }, + { + id: 'release:remove', + description: 'Remove a release from the config', + argNames: ['version'], + run: async (config, version) => { + await verifyWithInput(`Confirm you want to remove release: ${version} from the release config?`) + const rconfig = loadReleaseConfig() + removeScheduledRelease(rconfig, version) + saveReleaseConfig(rconfig) + }, + }, + { + id: 'release:activate-release', + description: 'Activate a feature release', + run: async config => { + await activateRelease(config) + }, + }, + { + id: 'release:deactivate-release', + description: 'Activate a feature release', + run: async config => { + await verifyWithInput('Are you sure you want to deactivate all releases?') + deactivateAllReleases(config) }, }, { @@ -899,10 +946,10 @@ ${patchRequestIssues.map(issue => `* #${issue.number}`).join('\n')}` description: 'Bake stitched migration files into the build for a release version. Only required for minor / major versions.', run: async config => { - const { upcoming } = await releaseVersions(config) + const release = await getActiveRelease(config) - const releaseBranch = `${upcoming.major}.${upcoming.minor}` - const version = upcoming.version + const releaseBranch = release.branch + const version = release.version.version ensureReleaseBranchUpToDate(releaseBranch) const prConfig = { @@ -948,6 +995,22 @@ ${patchRequestIssues.map(issue => `* #${issue.number}`).join('\n')}` rmdirSync(cacheFolder, { recursive: true }) }, }, + { + id: 'util:previous-version', + description: 'Calculate the previous version based on repo tags', + argNames: ['version'], + run: (config: ReleaseConfig, version?: string) => { + let ver: SemVer | undefined + if (version) { + ver = new SemVer(version) + console.log(`Getting previous version from: ${version}...`) + } else { + console.log('Getting previous version...') + } + const prev = getPreviousVersion(ver) + console.log(chalk.green(`${prev.format()}`)) + }, + }, { id: '_test:release-guide-content', description: 'Generate upgrade guides', @@ -971,11 +1034,12 @@ ${patchRequestIssues.map(issue => `* #${issue.number}`).join('\n')}` description: 'Test Google Calendar integration', run: async config => { const googleCalendar = await getClient() + const release = await getActiveRelease(config) await ensureEvent( { title: 'TEST EVENT', - startDateTime: new Date(config.releaseDate).toISOString(), - endDateTime: addMinutes(new Date(config.releaseDate), 1).toISOString(), + startDateTime: new Date(release.releaseDate).toISOString(), + endDateTime: addMinutes(new Date(release.releaseDate), 1).toISOString(), transparency: 'transparent', }, googleCalendar @@ -1025,7 +1089,7 @@ ${patchRequestIssues.map(issue => `* #${issue.number}`).join('\n')}` id: '_test:config', description: 'Test release configuration loading', run: config => { - console.log(JSON.stringify(config, null, ' ')) + console.log(JSON.stringify(config, null, 2)) }, }, { diff --git a/dev/release/src/util.ts b/dev/release/src/util.ts index 7ae816ff9e2..455008b7f39 100644 --- a/dev/release/src/util.ts +++ b/dev/release/src/util.ts @@ -64,17 +64,12 @@ async function readLineNoCache(prompt: string): Promise { export async function verifyWithInput(prompt: string): Promise { await readLineNoCache(chalk.yellow(`${prompt}\nInput yes to confirm: `)).then(val => { if (val !== 'yes') { - throw new Error() + console.log(chalk.red('Aborting!')) + process.exit(0) } }) } -export function getWeekNumber(date: Date): number { - const firstJan = new Date(date.getFullYear(), 0, 1) - const day = 86400000 - return Math.ceil(((date.valueOf() - firstJan.valueOf()) / day + firstJan.getDay() + 1) / 7) -} - export async function ensureDocker(): Promise> { return execa('docker', ['version'], { stdout: 'ignore' }) } @@ -206,13 +201,13 @@ export async function getContainerRegistryCredential(registryHostname: string): return credential } -export type ContentFunc = (previousVersion: string, nextVersion: string) => string +export type ContentFunc = (previousVersion?: string, nextVersion?: string) => string const upgradeContentGenerators: { [s: string]: ContentFunc } = { - docker_compose: (previousVersion: string, nextVersion: string) => '', - kubernetes: (previousVersion: string, nextVersion: string) => '', - server: (previousVersion: string, nextVersion: string) => '', - pure_docker: (previousVersion: string, nextVersion: string) => { + docker_compose: () => '', + kubernetes: () => '', + server: () => '', + pure_docker: (previousVersion?: string, nextVersion?: string) => { const compare = `compare/v${previousVersion}...v${nextVersion}` return `As a template, perform the same actions as the following diff in your own deployment: [\`Upgrade to v${nextVersion}\`](https://github.com/sourcegraph/deploy-sourcegraph-docker/${compare}) \nFor non-standard replica builds: @@ -275,3 +270,21 @@ export const updateUpgradeGuides = (previous: string, next: string): EditFunc => } } } + +export async function retryInput( + prompt: string, + delegate: (val: string) => boolean, + errorMessage?: string +): Promise { + while (true) { + const val = await readLine(prompt).then(value => value) + if (delegate(val)) { + return val + } + if (errorMessage) { + console.log(chalk.red(errorMessage)) + } else { + console.log(chalk.red('invalid input')) + } + } +} diff --git a/dev/release/templates/patch_release_issue_template.md b/dev/release/templates/patch_release_issue_template.md index 7a287a6d203..0a52667240a 100644 --- a/dev/release/templates/patch_release_issue_template.md +++ b/dev/release/templates/patch_release_issue_template.md @@ -27,7 +27,12 @@ This release is scheduled for **$RELEASE_DATE**. -- [ ] Ensure release configuration in [`dev/release/release-config.jsonc`](https://sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/dev/release/release-config.jsonc) on `main` is up-to-date with the parameters for the current release. +- [ ] Ensure release configuration in [`dev/release/release-config.jsonc`](https://sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/dev/release/release-config.jsonc) on `main` has version $MAJOR.$MINOR.$PATCH selected by using the command: + +```shell +pnpm run release release:activate-release +``` + - [ ] Ensure you have the latest version of the release tooling and configuration by checking out and updating `sourcegraph@main`. - [ ] Create a `Security release approval issue` and post a message in the [#security](https://sourcegraph.slack.com/archives/C1JH2BEHZ) channel tagging @security-support. @@ -109,14 +114,16 @@ Create and test the first release candidate: - [ ] Finalize and announce that the release is live: ```sh pnpm run release release:announce - pnpm run release release:close ``` ## Post-release -- [ ] Open a PR to update [`dev/release/release-config.jsonc`](https://github.com/sourcegraph/sourcegraph/edit/main/dev/release/release-config.jsonc) with the parameters for the current release. -- [ ] Ensure you have the latest version of the release tooling and configuration by checking out and updating `sourcegraph@main`. -- [ ] Let the #cloud team know about the managed instances upgrade issue. -- [ ] Close this issue. +- [ ] Close the release: + +```shell + pnpm run release release:close +``` + +- [ ] Open a PR to update [`dev/release/release-config.jsonc`](https://github.com/sourcegraph/sourcegraph/edit/main/dev/release/release-config.jsonc) after the auto-generated changes above. **Note:** If another patch release is requested after the release, ask that a [patch request issue](https://github.com/sourcegraph/sourcegraph/issues/new?assignees=&labels=team%2Fdistribution&template=request_patch_release.md) be filled out and approved first. diff --git a/dev/release/templates/release_issue_template.md b/dev/release/templates/release_issue_template.md index fcf052b7974..c4f1fc25b90 100644 --- a/dev/release/templates/release_issue_template.md +++ b/dev/release/templates/release_issue_template.md @@ -6,9 +6,8 @@ Arguments: - $MINOR - $PATCH - $RELEASE_DATE -- $ONE_WORKING_WEEK_BEFORE_RELEASE -- $THREE_WORKING_DAY_BEFORE_RELEASE -- $ONE_WORKING_DAY_AFTER_RELEASE +- $SECURITY_REVIEW_DATE +- $CODE_FREEZE_DATE --> # $MAJOR.$MINOR release @@ -19,14 +18,19 @@ This release is scheduled for **$RELEASE_DATE**. ## Setup -- [ ] Ensure release configuration in [`dev/release/release-config.jsonc`](https://sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/dev/release/release-config.jsonc) on `main` is up to date with the parameters for the current release. +- [ ] Ensure release configuration in [`dev/release/release-config.jsonc`](https://sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/dev/release/release-config.jsonc) on `main` has version $MAJOR.$MINOR.$PATCH selected by using the command: + +```shell +pnpm run release release:activate-release +``` + - [ ] Ensure you have the latest version of the release tooling and configuration by checking out and updating `sourcegraph@main`. -## Security review (one week before release - $ONE_WORKING_WEEK_BEFORE_RELEASE) +## Security review (one week before release - $SECURITY_REVIEW_DATE) - [ ] Create a [new issue](https://github.com/sourcegraph/sourcegraph/issues/new/choose) using the **Security release approval** template and post a message in the [#security](https://sourcegraph.slack.com/archives/C1JH2BEHZ) channel tagging `@security-support`. -## Cut release (three days before release - $THREE_WORKING_DAY_BEFORE_RELEASE) +## Cut release (three days before release - $CODE_FREEZE_DATE) Perform these steps three days before the release date to generate a stable release candidate. @@ -134,29 +138,19 @@ On the day of the release, confirm there are no more release-blocking issues (as - [ ] Cherry pick the release-publishing PR from the release branch into main - [ ] Alert the marketing team in [#release-post](https://sourcegraph.slack.com/archives/C022Y5VUSBU) that they can merge the release post. - [ ] Finalize and announce that the release is live: - ```sh pnpm run release release:announce ``` ### Post-release -- [ ] Notify the next release captain that they are on duty for the next release. -- [ ] Open a PR to update [`dev/release/release-config.jsonc`](https://sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/dev/release/release-config.jsonc) with the parameters for the next release. - - [ ] Change `upcomingRelease` to the current patch release - - [ ] Change `previousRelease` to the previous patch release version - - [ ] Change `releaseDate` to the current date (time is optional) along with `oneWorkingDayAfterRelease` and `threeWorkingDaysBeforeRelease` - - [ ] Change `captainSlackUsername` and `captainGitHubUsername` accordingly -- [ ] Ensure you have the latest version of the release tooling and configuration by checking out and updating `sourcegraph@main`. -- [ ] Create release calendar events, tracking issue, and announcement for next release: - +- [ ] Create release calendar events, tracking issue, and announcement for next release (note: these commands will prompt for user input to generate the definition for the next release): ```sh pnpm run release tracking:issues pnpm run release tracking:timeline ``` - +- [ ] Open a PR to update [`dev/release/release-config.jsonc`](https://sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/dev/release/release-config.jsonc) with the auto-generated changes from above. - [ ] Close the release. - ```sh pnpm run release release:close ``` diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 3fd9d571063..3453d5e74c5 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -1182,7 +1182,12 @@ importers: '@sourcegraph/testing': link:../testing dev/release: - specifiers: {} + specifiers: + '@types/luxon': ^3.2.0 + luxon: ^3.2.1 + dependencies: + '@types/luxon': 3.2.0 + luxon: 3.2.1 schema: specifiers: {}