Use [esbuild](https://esbuild.github.io/) instead of Webpack for builds of `client/web`, for faster builds (dev and prod) and greater dev-prod parity. This PR completely removes all use of Webpack in this repository.
`client/web` is the last build target that still uses Webpack; all others have been recently migrated to esbuild. Most devs here have been using esbuild for local dev of `client/web` for the last 6-12 months anyway. The change here is that now our production builds will be built by esbuild.
All sg commands, integration/e2e tests, etc., continue to work as-is. The bundlesize report will take a while to stabilize because the new build products use different filenames.
## Benchmarks
Running `pnpm run generate && time pnpm -C client/web run task:gulp webBuild` and taking the `time` output from the last command:
- Webpack: 62.5s
- esbuild: 6.7s
Note: This understates esbuild's victory for 2 reasons: (1) because esbuild is building both the main and embed entrypoints, whereas Webpack only builds the main entrypoint in this benchmark) and (2) because a lot of it is in the fixed startup time of `gulp`; esbuild incremental rebuilds during local dev only take ~1s.
## Notes
We no longer use Babel to produce web builds (we use esbuild), so we don't need any Babel plugins that optimize the output or improve browser compatibility. Right now, Babel is only used by Jest (for tests) and by Bazel as an intermediate step.
This file is produced by both Webpack and esbuild. It is not Webpack-specific.
```
fastmod -e yaml,ts,js,md,tsx WEBPACK_MANIFEST WEB_BUILD_MANIFEST
fastmod -e yaml,ts,js,md,tsx,go webpack.manifest.json web.manifest.json
fastmod -e yaml,ts,js,md,tsx,go webpackManifestPath webBuildManifestPath
fastmod -e yaml,ts,js,md,tsx,go 'webpack manifest' 'web build manifest'
fastmod -e go,ts,tsx webpackManifest webBuildManifest
fastmod -e go,ts,tsx WebpackManifest WebBuildManifest # not all accepted
```
This is the end of the PR train to remove the enterprise directory from out repo since we have consolidated to use a single license.
Bye rough code split :)
* Add note to gitignore about cody directory inclusion
* Update docstring about user-history usage
* Re-index user history entries by user ID
* Pass authenticated user ID to useUserHistory hook
* Pass `authenticatedUser` through Cody chat UI component props
* Refactor scope selector queries to use fragments
* Hook up repo selector popover to query 10 most recently visited repos
* Change visibility of `UserHistory.loadEntries()`
* Pass authenticated user through a couple more components
* Refactor suggested repos query to grab 10 by name and first 10 embedded
* Construct suggested repos list from user history, transcript history, and defaults
* Display suggestions, cleanup popover height/display
* Initialize new transcript with previous scope
* Type fixups
* Some language cleanup
* Add docstring for user history hook
* Refactor repo suggestions out to a custom hook
* Re-index Cody transcript history entries by user ID
* Don't show suggestions section at all if there are none
* Update for autoload case
* Update cody-shared dependency
* Fix placeholder spacing
* DRY up suggestions rendering, improve spacing
* Use abbreviated repo name in header
* Bazel configure
* Update editor implementations for cody-shared package
* Add extra top padding to section headers
* Vertically align radio buttons in rows
* Update unindexed warning copy, link color
* Hide context explanation when repos are selected
* Remove "Great"
Reduce polling interval to 2s from 5s for embedding status indicator.
## Test plan
- sg start dotcom
- visit /cody
- add repos to context
- check network tab for repos embeddings status gql request every 2
seconds.
Because we at least need one file to be present or the embed won't work,
this replaces CHANGELOG.md by CONTRIBUTING.md as a placeholder. This
will prevent PRs updating the changelog from invalidating the cache for
nothing for these slow targets.
## Test plan
<!-- All pull requests REQUIRE a test plan:
https://docs.sourcegraph.com/dev/background-information/testing_principles
-->
CI
Re-organizing a couple of things to make the individual files nicer,
consolidated some things in ctags.nix, extracted forEachSystem stuff to
flake.nix instead of the individual files, overlay for our specific
nodejs requirements, some more comments to stuff
## Test plan
N/A, nix stuff
Use volatile status variables provided by stamping to inject relevant
Git information into the Percy execution process. Percy needs Git commit
and branch to create an accurate report and auto-accept changes on main.
See extensive comments in the code for implementation details.
On the high level, instead of running the Percy CLI directly from Bazel,
we wrap it into the JS script that:
1. Reads volatile status variables with Git info.
2. Creates a Percy CLI command with these variables injected.
3. Exectes the command using args received from the Bazel target.
## Test plan
1. CI
2. Locally: `bazel test //client/web/src/integration:integration-tests
--define=E2E_HEADLESS=true
--define=E2E_SOURCEGRAPH_BASE_URL="http://localhost:7080"
--define=GH_TOKEN=fake --define=XXX` — replace `XXX` with the
`PERCY_TOKEN` from our shared 1Password vault.
Some sprinkling of special sauce for the NixOS (and nix shell) users in
the house. Much care has been taken so that there should be no
interference with anyone running bazel on a non-nixos machine, and that
it also works as (mostly) expected for nix shell users on macos.
## Test plan
Ran main dry-run, all green, so it shouldnt interfere with anyone on a
non-nixos machine
Clones and patches Sourcegraph git repository under
@sourcegraph_back_compat so we can run tests targets from that
particular release against the new database schema.
Flakes can be defined to skip known problematic tests which are either
flaky or simply cannot run against the new schema in case of a breaking
change. See //dev/backcompat:flakes.bzl for more details about how to
define them.
The final result is the definition of a @sourcegraph_back_compat target,
whose test targets are exactly the same as back then, but with instead a
new schema.
Example: `bazel test
@sourcegraph_back_compat//enterprise/internal/batches/...`.
See
https://github.com/sourcegraph/sourcegraph/pull/50932/files#diff-2f07315ec320aa4080768fec54f32ebb2cbf4e3e6df7c51a314beda827c48c41R104
for the command generating the mandatory patch file in CI for these
tests to run.
If the patch file were to be missing, a placeholder diff is in place to
make it explicit (in the eventuality of someone running those locally).
A new CI job has been added because this one is fully independent from
the other targets to build, and will be cached most of the time.
Depending how it goes, I might bring that one over the main bazel job to
avoid getting an agent just for that.
Because it's cached, we can run this within PRs and have it take 10s max
for 99% of the PRs and main builds.
TODO: The flakefiles are still there, they're just defined in different
place. I need to port the annotation as well, that has proven to be
really useful.
Also right now it's a bit manual as the 5.0.0 target is a bit peculiar,
but that'll get much simpler as we progress, as the next old release
will be requiring less and less patching.
This turns about a dozen of 10m individual jobs into a single one
ranging from 30s if there's not database changes to 10m if there's some
of them and cold cache.
## Test plan
<!-- All pull requests REQUIRE a test plan:
https://docs.sourcegraph.com/dev/background-information/testing_principles
-->
QA: fails on old code when a new migration breaks something:
-
https://buildkite.com/sourcegraph/sourcegraph/builds/214283#01879f2d-67be-4caf-b5d5-93f045e19348/118-126
* internal: add service and singleprogram packages
* sg.config.yaml: add single-binary build targets
* internal/env: add a function for clearing environ cache
* internal/{workerutil,metrics}: add a hack to allow running 2 executors in the same process
* internal/conf: add single-program deploy type
* internal/singleprogram: clarify security
* cmd/sourcegraph-oss: add initial single-binary main (will not build yet)
* enterprise/cmd/sourcegraph: initial enterprise single-binary
* Add multi-platform builds for single-program
* single-binary: correctly build JS artifacts into binary
* license_finder licenses add github.com/xi2/xz "Public domain"
* internal/service/svcmain: correctly initialize logger for DeprecatedSingleServiceMain
* worker: refactor to new service pattern
* cmd/github-proxy: refactor to use new service pattern
* symbols: refactor to use new service pattern
* gitserver: refactor to user new service pattern
* searcher: refactor to use new service pattern
* gitserver: refactor to use new service pattern
* repo-updater: refactor to use new service pattern
* frontend: refactor to use new service pattern
* executor: refactor to use new service pattern
* internal/symbols: use new LoadConfig pattern
* precise-code-intel-worker: refactor to use new service pattern
* internal/symbols: load config for tests
* cmd/repo-updater: remove LoadConfig approach
* cmd/symbols: workaround env var conflict with searcher
* executor: internal: add workaround to allow running 2 instances in same process
* executors: add EXECUTOR_QUEUE_DISABLE_ACCESS_TOKEN for single-binary and dev deployments only
* single-binary: use EXECUTOR_QUEUE_DISABLE_ACCESS_TOKEN
* extsvc/github: fix default value for single-program deploy type
* single-binary: stop relying on a local ctags image
* single-binary: use unix sockets for postgres
* release App snapshots in CI when pushed to app/release-snapshot branch
* internal/service/svcmain: update TODO comment
* executor: correct DEPLOY_TYPE check
* dev/check: allow single-binary to import dbconn
* executor: remove accidental reliance on dbconn package
* executor: improve error logging when running commands (#46546)
* executor: improve error logging when running commands
* executor: do not attempt std config validation running e.g. install cmd
* executor: do not pull in the conf package / frontend reliance
* ci: executors: correct site config for passwordless auth
* server: fix bug where github-proxy would try to be a conf server
* CI: executors: fix integration test passwordless auth
* executors: allow passwordless auth in sourcegraph/server for testing
* repo-updater: fix enterprise init (caused regression in repository syncing)
Signed-off-by: Stephen Gutekanst <stephen@sourcegraph.com>
Co-authored-by: Peter Guy <peter.guy@sourcegraph.com>
Co-authored-by: Quinn Slack <quinn@slack.org>
Bazel's rules_js rely on the pnpm package manager. To simplify the integration, we're migrating to pnpm from our current package manager — yarn. Another reason to migrate is that pnpm is cool and fast. 😉
ignore go.work.sum in addition to go.work
This adds go.work.sum to the .gitignore in addition to go.work so that
it is not accidentally committed. The generated file is only relevant to
a local dev env.
This is a new feature in go1.18 to simplify working on multiple modules.
For example I can easily test sourcegraph with local changes to zoekt by
running the command:
go work init . ../../google/zoekt/
These files shouldn't be committed, as they are files that are specific
to someones task.
Test Plan: my go.work file doesn't show up in git status
Integrates test reports scraping into AnnotatedCmd, removing a bunch of duplication and complexity from our test scripts and packaging test uploads into a neat pull-based API that allows local inspection of artefacts similarly to the annotations API.
bk.AnnotatedCmd is the generator-based pull-oriented annotations API to replace sporadic calls to annotate.sh and buildkite-agent annotate, splitting out annotation creation concerns out of check scripts and into the pipeline generator.
