Reimplement fetching & building code-intel-extension bundle as a bazel `http_archive` repository & refactored `js_run_binary`. This should allow the extensions bundle to be cached in the repository cache, instead of being affected by changing action environments
## Test plan
`bazel build //client/browser:code-intel-extensions` and `cd client/browser && pnpm run create-source-zip` run without error
This disables the bundling of code intel extensions with the firefox browser extension. The bundled files are minified, and I failed to run the build steps for the extensions locally (they're pretty out of date), so I can't really test the changes needed to build a non-minified version.
For now, rather than spending more time on extensions that live in a hazy state of support, this just removes the extensions from the firefox addon so we don't get removed from the store.
Use [esbuild](https://esbuild.github.io/) instead of Webpack for builds of `client/web`, for faster builds (dev and prod) and greater dev-prod parity. This PR completely removes all use of Webpack in this repository.
`client/web` is the last build target that still uses Webpack; all others have been recently migrated to esbuild. Most devs here have been using esbuild for local dev of `client/web` for the last 6-12 months anyway. The change here is that now our production builds will be built by esbuild.
All sg commands, integration/e2e tests, etc., continue to work as-is. The bundlesize report will take a while to stabilize because the new build products use different filenames.
## Benchmarks
Running `pnpm run generate && time pnpm -C client/web run task:gulp webBuild` and taking the `time` output from the last command:
- Webpack: 62.5s
- esbuild: 6.7s
Note: This understates esbuild's victory for 2 reasons: (1) because esbuild is building both the main and embed entrypoints, whereas Webpack only builds the main entrypoint in this benchmark) and (2) because a lot of it is in the fixed startup time of `gulp`; esbuild incremental rebuilds during local dev only take ~1s.
## Notes
We no longer use Babel to produce web builds (we use esbuild), so we don't need any Babel plugins that optimize the output or improve browser compatibility. Right now, Babel is only used by Jest (for tests) and by Bazel as an intermediate step.
* use single-file entrypoints for browser ext webpack config
This is more standard and puts more of the config in code not config. This also makes it easier to support esbuild builds for the browser extension (esbuild does not support multiple sources for an entrypoint, because such behavior is undefined except by webpack, and who knows what it does).
* use different file basenames for native & phabricator integrations
This makes it easier (in the future) to simplify our bundler config for the browser extension because some bundlers (esbuild via Bazel, for example) do not support 2 entrypoints with the same file basename.
* name entrypoint bundles after their entrypoint, not a custom name
This removes another source of "names" and also makes it so that the browser extension can be built by esbuild via Bazel in the future (which does not support renaming entrypoints).
Also remove Webpack auto-reloading because its implementation depended on this. There is a workaround, but the intent is to remove Webpack for bext builds, so it's not worth keeping it.
This PR adds the eslint rules @typescript-eslint/consistent-type-imports and @typescript-eslint/no-import-type-side-effects. Having type imports be explicitly declared as such makes it easier for modern bundlers to strip those imports which is useful if e.g. a module is sent directly to the browser.
I added both rules as a warning because @typescript-eslint/consistent-type-exports is also a warning.
1. Adds `//client/browser:bundle` for the browser extension build.
2. Adds `//client/browser:code-intel-extensions` for the inlined code
extension logic.
3. Adds `//client/browser:integration-assets` for the browser extension
integration assets which should be added to each sourcegraph image.
4. Adds `client/browser/config/webpack/base.config.bazel.ts` in the same
way we do it for the web application — duplicated webpack config
specifically for Bazel with the idea to delete non-Bazel one later.
5. Restructures browser extension `config` folder to make it
Bazel-friendly.
6. Instead of
[sourcegraph-extension-bundles-revision.txt](https://github.com/sourcegraph/sourcegraph/pull/51135/files#diff-eed06bee70ccc222acfa612bdcd09a2417f01072da91b3de1a76611b2ef44d53)
we now use function arguments for revision numbers.
7. Adds browser integration assets target to UI assets.
## Test plan
1. CI
2. `bazel build //client/browser:bundle`
3. `bazel build //client/browser:integration-asssets`
4. `bazel build //ui/assets/oss:copy_bundle`
5. `bazel build //ui/assets/enterprise:copy_bundle`
Bazel's rules_js rely on the pnpm package manager. To simplify the integration, we're migrating to pnpm from our current package manager — yarn. Another reason to migrate is that pnpm is cool and fast. 😉
Part of #41921
We have a deployment method of the browser extensions that we call
"native integrations". The idea here is that we inject the browser
extension code directly from the code host so that users do not need to
install an extension. Most prominently, this is used by GitLab who
currently bundles a version of the native integrations package with
their on-premise builds so instance admins can enable this for users.
The issue with this deployment model is that we have no impact on when
these clients are updated. We rely on GitLabs rollout and update
policies so these cycles are super slow. For the upcoming release, we
had to cut a corner for this and made the extensions GraphQL endpoints
handle eventual native integration requests with special care to not
break them.
Since we eventually want to remove these APIs, we want to move fast here
and provide a new native integration build that does not use these APIs
anymore. Before we can update the bundled version on the GitLab end, we
need to make sure that the package contains a bundled version of the
code intel APIs (similar to our browser extensions right now). This is
what's happening here:
- This PR adds bundles the legacy code intel APIs to the
`@sourcegraph/code-host-integration` package.
- It also changes the gating of the inlined extensions so that they are
loaded on GitLab.
Co-authored-by: Taras Yemets <yemets.taras@gmail.com>
Recently, the end-to-end tests have been failing due to variability in GraphQL API responses. This motivated us to create a more controlled testing environment for the browser extension relative to end-to-end tests, but that still run in a browser against real code hosts, unlike our unit test suite.
The integration tests introduced in this PR validate that Sourcegraph commits don't break the browser extension ahead of time, not 0-24 hours after a regression is introduced. Since these tests are run against (Polly.JS) recordings of code hosts and use mock data for our backend, they complete fast enough to not require nightly tests.
- web: upgrade `Webpack` to v5 for the web app
- web: upgrade `Webpack` to v5 for Storybook
- web: upgrade `Webpack` to v5 for the browser extension
- web: enable filesystem cache only in the development environment
- web: disable node.js polyfills
- web: disable `webpack.IgnorePlugin` for the web app
- web: switch to `react-refresh-webpack-plugin` from `react-hot-loader`
- web: use `cache` only in development mode
- web: remove redundant polyfill dependencies
- web: do not use `Webpack` cache on CI
- web: use runtime bundle in development mode by `frontend` server
* Use browser API
* Renames
* Polyfill globalThis in Jest
* Don't use dynamic import
* More type safe storage access
* Prettier webpack log
* Don't use NonNullable
* Remove unused background page handlers
* Replace sourcegraph.com string literals with constant
* Use BackgroundMessageHandlers interface in client too
* Stricter tsconfig for the browsr extension
* Refactor getPhabricatorState() to async function
This was the best way to fix the many noImplicitReturns violations in this function.
* Fix Alerts.tsx
* Update client/browser/src/shared/repo/backend.tsx
Co-Authored-By: lguychard <l.guychard@gmail.com>
* `| null` -> `| undefined`
* Add return type
