What: This PR does the bare minimum to migrate the current community
search pages to Svelte. A better strategy for managing them is needed in
the medium/long term.
How: The community pages live at the root (e.g. `/kubernetes`) which
complicates things, but I'll get to that later. The page is implemented
as a single parameterized route. A parameter matcher is used to validate
the community name. Because these pages should only be accessible on
dotcom the matcher also validates whether or not we are on dotcom (if
not, the path will be matched against a different route).
The page config is stored in a separate module so that it's no included
in every page and so that it can be used in the integration test.
The loader and page implementation themselves are straightforward. I
made a couple of changes in other modules to make implementation easier:
- Extracted the parameter type of the `marked` function so that it can
be used as prop type.
- Added an `inline` option to `marked` that allows formatting markdown
as 'inline', i.e. without `p` wrapper.
- Added a `wrap` prop to `SyntaxHighlightedQuery.svelte` to configure
line wrapping of syntax highlighted search queries (instead of having to
overwrite styles with `:global`).
- Extended the route code generator to be able to handle single
parameter segments and the `communitySearchContext` matcher.
Because the community routes should only be available on dotcom I added
a new tag to the code generator that allows it include routes only for
dotcom.
Once we change how all this works and have community search pages live
under a different path we can simplify this again.
Result:
| React | Svelte |
|--------|--------|
|
|
|
## Test plan
- New integration tests.
- Verified that `/kubernetes` shows a 'repo not found error' when
running against S2.
- Verified that `/kubernetes` shows the community page when running
against dotcom.
- Verified that `window.context.svelteKit.enabledRoutes` contains the
community page route in enterprise mode but not in dotcom mode.
This adds support for navigating between search results with keyboard
shortcuts. Similar to the React app, `j` or `down` means "next result",
and `k` or `up` means previous results. To accompany this change, when a
search is submitted, the first result is focused by default to
facilitate iterating over results with the keyboard.
The first version of the file tree for the revision panel had flat file
names. This meant that files were not organized, they were very
horizontal-space-sensitive, and you could not filter to a directory
(only repo and file name).
This updates the file filter to be a full tree, which I find much easier
to use.
Required to build an updated subscriptions management UI.
Most of the diff is generated proto for some reason
Closes https://linear.app/sourcegraph/issue/CORE-226
## Test plan
Integration tests
Backcompat test suite works by checking out the code at the previous
minor release, injects the database schema from `HEAD` and run all Go
tests. This ensures that the old code can run against the new schema,
thus being backward compatible.
We forgot to update this the last time, that's why I'm bumping by two
tags here.
## Test plan
CI
This reverts commit 43e06cef0c.
This migration deadlocked on S2, we didn't expect that we have processes
taking table locks but apparently so.. Reverting for now and will
manually fix up S2 once the build went through. We probably need to make
this one migration per table :rip:
## Test plan
Revert.
Previously, `sg` would give no notice that an auto-update of itself was
happening in the background, due to background output being buffered.
This would be confusing when an invocation hangs/doesnt terminate as
quickly as expected due to the update process still being in progress.
Instead, we should print at least certain output from the process
immediately
## Test plan
`go run ./dev/sg -- <some command>` with a time.Sleep
## Changelog
<!-- OPTIONAL; info at
https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c
-->
Connecting to the wrong DSN leads to a cryptic error:
```
Message: {"SeverityText":"FATAL","Timestamp":1723196934012096886,"InstrumentationScope":"init db (syntactic-codeintel-worker)","Caller":"shared/shared.go:87","Function":"github.com/sourcegraph/sourcegraph/cmd/syntactic-code-intel-worker/shared.initCodeintelDB","Body":"Failed to connect to codeintel database","Resource":{"service.name":"syntactic-code-intel-worker","service.version":"286647_2024-08-08_5.6-34a7914fb884","service.instance.id":"syntactic-code-intel-worker-7bb9ccc75c-mkzpk"},"Attributes":{"error":"database schema out of date"}}
```
## Test plan
- existing tests should continue to pass
<!-- REQUIRED; info at
https://docs-legacy.sourcegraph.com/dev/background-information/testing_principles
-->
## Changelog
<!-- OPTIONAL; info at
https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c
-->
In https://github.com/sourcegraph/sourcegraph/pull/63946 we reworked how
we handle assets, but we overlooked updating the target in
`sg.config.yaml`.
## Test plan
CI + locally tested (made some changes on the home page, got reloaded as
expected)
Fixes CODY-3081
This is the first PR for the project [⏭️ Launch Cody API
Experimental](https://linear.app/sourcegraph/project/launch-cody-api-experimental-8fd5ec338bf4),
which falls under the umbrella of moving Cody's brains to the cloud.
Previously, there was no publicly available REST API for our customers
to interact with Cody. This is a frequently requested feature from
customers and prospects.
This PR adds a new `POST /api/v1/chat/completions` endpoint, which
should be compatible with existing OpenAI clients. The OpenAI API format
is increasingly becoming an industry standard so this seems like a good
first step towards exposing a stable publicly facing API for our
customers.
The goal is to add more Cody-specific APIs in the coming weeks to send
chat messages and reference context.
<!-- PR description tips:
https://www.notion.so/sourcegraph/Write-a-good-pull-request-description-610a7fd3e613496eb76f450db5a49b6e
-->
## Test plan
See added test cases.
<!-- REQUIRED; info at
https://docs-legacy.sourcegraph.com/dev/background-information/testing_principles
-->
## Changelog
* API: new publicly available `/api/v1/chat/completions` REST endpoint
that is compatible with OpenAI clients with some restrictions. The detailed list of restrictions will eventually be documented on sourcegraph.com/docs
<!-- OPTIONAL; info at
https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c
-->
We introduce the tenant_id column to every table without any magic
enforcement or magic default value. Instead we default to NULL and plan
to have an out of band migration. From out testing this change is
instant since it is only a table metadata change. According to the
documentation this sort of ALTER TABLE is instant since Postgres 11.
Test Plan: ran this migration against a clone of the s2 database.
---------
Co-authored-by: Keegan Carruthers-Smith <keegan.csmith@gmail.com>
Currently, this uses context.Background, because the initial context
might've been canceled.
However, that causes us to lose any context involved here, so traces
won't include this query and tenant information would be lost.
This uses the new-ish context.WithoutCancel to avoid this issue and
stores the context used to create the transaction to close it later.
Test plan: Ran a local instance and didn't see errors, also integration
and E2E tests pass.
To ease the transition from older search semantics to keyword search, we
[boost matches on
phrases](https://github.com/sourcegraph/sourcegraph/pull/59940). This
phrase boosting applied to all search types, including repo, and symbol
searches. However, these search backends don't effectively handle the
extra boosted OR clause, so it can hurt performance and ranking.
This PR limits the phrase boosting to text searches (`type:file` and
`type:path`), as they benefit most from the boosting and can handle it
efficiently.
closes:
https://linear.app/sourcegraph/issue/SRCH-622/remove-old-cody-web-entirely
Removes the old Cody Web completely.
## Test plan
- sg start
- visit /cody/chat and you should see the new unified panel Cody Web
without having to turn on any feature flags.
- check the blob view sidebar for the same.
## Changelog
- The new Cody Web is enabled by default and the older version of Cody
Web is completely removed from both standalone chat and sidebar.
Some analysis phase performance improvements in rules_js 2.0.0-rc9 that
are worth picking up. rules_js is very close to 2.0.0 final now. Waiting
on one last improvement requiring and API change for bzlmod.
## Test plan
CI
## Changelog
On double-checking the various Cody Gateway access management CRUD I
noticed that I did not implement the ability to remove the override
entirely properly - right now, removing a rate limit with `field_mask`
and no values results in a zero-value rate limit rather than a null one
as desired.
This change makes it so that if you provider a nil `*_rate_limit`
object, all field paths for that object is also set to nil, removing the
override and allowing the default active-license-plan-based rate limits
to apply.
This also adds a minor adjustment to Cody Gateway to correctly handle
zero durations for rate limit intervals, which should be treated as "no
access".
## Test plan
Unit/integration tests, and a manual test:
- Check out https://github.com/sourcegraph/sourcegraph/pull/64090 and
run `sg start dotcom`
- Go to a subscription, add a rate limit override
- Click the 🗑️ button to remove the rate limit override
- Repeat above on each property
Sourcegraph's GitHub discussions have been defunct for awhile, but links
to them persisted in READMEs and documentation.
## Test plan
Documentation update only.
This change follows
https://github.com/sourcegraph/sourcegraph/pull/63858 by making the Cody
Access APIs _read_ from the Enterprise Portal database, instead of
dotcomdb, using the data that we sync from dotcomdb into Enterprise
Portal.
As part of this, I also expanded the existing "compatibility" test suite
that compares the result of our dotcomdb queries against the existing
GraphQL resolvers in dotcom to also compare the results of our new Cody
Access APIs, to validate that they return the same access.
> [!WARNING]
> There is one behavioural change, which is that hashes of _expired
licenses_ will no longer be valid as access tokens. This shouldn't be an
issue if customers use zero-config (implied access token from their
license key) - I will do some outreach before rolling this out.
Subsequent PRs will implement write APIs.
Part of https://linear.app/sourcegraph/issue/CORE-218
Part of https://linear.app/sourcegraph/issue/CORE-160
## Test plan
Integration and unit tests at various layers
This adds a new profile output for when a sql query does not have a
tenant set. This can be downloaded like other pprof profiles and
inspected by "go tool pprof". This allows us to easily see the most
common stack traces and sources of missing context.
Test Plan: downloaded from the debugserver the new profile type and
inspected with "go tool pprof"
---------
Co-authored-by: Erik Seliger <erikseliger@me.com>
Fixes srch-867
This commit adds a new command, `web-sveltekit-prod`, which simplies
builds the SvelteKit app and exits.
This command is now run by default when using `sg start`, so the assets
will be available locally.
Note however that the build process is quite slow, which is why I didn't
make it rebuild on file changes. Using the vite dev server, e.g. via `sg
start enterprise-sveltekit`, delivers a much better experience.
To make the integration a bit more ergonomic, I made additional changes:
- When run via `sg` (determined by checking `DEPLOY_TYPE`) compiler
warnings are not printed. That reduces noise in the `sg` output.
- I made similar changes to the svelte check bazel target to make it
easier to spot actuall errors (warnings are still important to fix but
since they don't cause the target to fail they just make it difficult to
spot the actual error).
- Since I finally found out about the `onwarn` handler, I made it so
that warnings about missing declarations are ignored. These warnings
occur for every icon because the Svelte compiler doesn't respect ambient
d.ts files.
- And since I made icon warning related changes I also documented a bit
how icons work in the app.
## Test plan
- `sg start` runs the `web-sveltekit-prod` command and the Svelte app is
served when going to `https://sourcegraph.test:3443/search`.
- The output of `bazel test //client/web-sveltekit:svelte-check` now
only contains errors.
- The output of `pnpm dev` (vite devserver) shows warnings that are not
related to icons.
Fixes srch-859
Problem: Currently the sidebar can reopen in the following scenario:
- Go to search home and submit a search
- On the search results page click the menu icon to open the sidebar and
click 'Code Search' to go back to search home.
- Execute a different query
- -> this brings you back to search results but with the menu sidebar
open
Looking at the code it's understandable that this happens: We never
close the sidebar. But this problem wasn't very visible because every
entry in the nav sidebar either brings you to the React app or to a page
with inline nav menu.
So this commit fixes the issue by closing the sidebar whenever we
navigate to a different page.
## Test plan
Manual testing. I added a temporary nav entry that would bring me to the
sourcegraph repo page. Clicking on it from the search results page
caused the sidebar to close.
Give a more descriptive error when we are unable to find a merge base
and can't find the files that have changed.
Closes DINF-162
## Test plan
1. `git fetch -v --prune --depth=100 -- origin
ccbaba24b72f3c6f4524b3f560ca839143ea463b`
2. `git merge-base HEAD origin/main` --- you'll get nothing since there
is no merge base in the current history
Increase the repo history
1. `git fetch --unshallow`
2. `git merge-base HEAD origin/main`
```
0a6e509af3
```
## Changelog
<!-- OPTIONAL; info at
https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c
-->
Co-authored-by: Bolaji Olajide <bolaji.olajide@sourcegraph.com>
In some deployments, these are running in separate instances. This
migration makes sure that we can still reference the tenants from the
tenant_id column reliably everywhere.
It's critical that this migration is a noop on instances that already
have this table, so please double check me here during code review.
Test plan: Integration and E2E tests still pass, as this should be a
noop.
Unless tenants are enforced, we use the old code path. If they are
enforced:
- zips are stored in `<cache-dir>/tenants/<tenantID>/<key>`
- evict runs globally
I also added another test helper to create tenants with fresh IDs for
every call
## Test plan:
- Updated unit tests
- Manual testing: I verified that unindexed symbol and text search work
and that a `/tenant/1` dir is created in both cache dirs
(`/tmp/searcher-archive` and `/tmp/symbols-cache`).
The utils are currently used outside of cmd/frontend, so to break the
internal -> cmd import I'm moving and merging the utils package with
internal/gqlutil.
Test plan: Just moved around and renamed a package, the go compiler
doesn't complain.
Now that https://github.com/sourcegraph/sourcegraph/pull/64339 is
merged, we can tell users more about what to expect with `sg start
single-program-experimental-blame-sqs`. And as it's been in flight for a
while now, it's safe to say that's it's time to give it a shorter name
😊.
So it's been renamed from `single-program-experimental-blame-sqs` to
`minimal`. And to ensure nobody is getting confused, a `deprecated`
attribute has been added on the command sets, which is used here to
indicate that the new alternative is `sg start`.
❓ Thoughts about `sg start minimal`? `sg start single` perhaps?
Running the old commandset: (ignore the yellow message, that's just a
local warning from `sg`)
Running the new commandset, with the preamble explaining what to expect:
## Test plan
Locally tested.
Fixes srch-858
Currently the menu icon rotates when hovering over it. The animation
should only target the Sourcegraph logo.
Additional changes:
- Move icon color style prop into CSS declaration (avoids inserting an
additional element)
- Let the animation target the link instead of the icon. There doesn't
seem to be reason why we actually need to target the `svg` element via
`:global`.
## Test plan
Manual testing.
This PR adds a basic table for storing tenants in the database, and automatically inserts a tenant 1 for existing instances.
Test plan: Adds a new, not yet used table. The migration is very simple, so shouldn't cause any issues to existing instances.
This PR aims to craft the /internal/tenant package for use by all Sourcegraph cluster-internal services to properly scope data visibility to the correct requesting tenant.
For now, we only expose methods we know we will DEFINITELY need.
This PR also adds the required middlewares so we can start to tinker with it in implementations.
## Test plan
CI passes. We don't enforce anything for now except not passing unparseable tenant IDs, which should be fine.
There currently isn't a 'one step' way to start a local Sourcegraph
instance with the SvelteKit app.
This commit adds `sg start enterprise-sveltekit` to fix that.
The changes in https://github.com/sourcegraph/sourcegraph/pull/64272
allow us to run the vite dev server in front of the Sourcegraph
instance, instead of building the assets and having them served by
frontend.
This gives us the benefit of hot module reloading and in general seems
to be a less fragile approach.
It's basically the same what we do with the React app in development
mode.
## Test plan
`sg start enterprise-sveltekit` starts the vite dev server as well as
the sourcegraph instance. Navigating to
`https://sourcegraph.test:3443/search` opens the Svelte app (when
enabled and logged in). Making a change in a source file updates the web
page immediately.
`sg start web-sveltekit-standalone` still works
I don't know why but having `overflow-x` causes the content of the
hovercard to not fully expand to the edges. It has something to do with
the sizing of the `hr` element.
## Test plan
Manual testing.
There was a todo comment that said we want to consolidate them but didn't yet to keep another diff smaller - so now we're doing that.
Test plan: Integration tests for search still pass.
To prevent cross-cmd imports in the future, moving the backend package into internal.
Test plan: Just moved a package around, Go compiler doesn't complain and CI still passes.
These functions are not required to be called outside of frontend, so there's no need to reexport them. Instead, we consolidate the signout cookie logic in the session package.
Test plan: Just moved some code around, go compiler doesn't complain.
