**Public saved searches will let us make global saved searches for
dotcom and for customers to help them discover and share awesome search
queries!**
Saved searches now have:
- Visibility (public vs. secret). Only site admins may make a saved
search public. Secret saved searches are visible only to their owners
(either a user, or all members of the owning org). A public saved search
can be viewed by everyone on the instance.
- Draft status: If a saved search's "draft" checkbox is checked, that
means that other people shouldn't use that saved search yet. You're
still working on it.
- Timestamps: The last user to update a saved search and the creator of
the saved search are now recorded.
Also adds a lot more tests for saved search UI and backend code.
## Test plan
Create a saved search. Ensure it's in secret visibility to begin with.
As a site admin, make it public. Ensure other users can view it, and no
edit buttons are shown. Try changing visibility back and forth.
## Changelog
- Saved searches can now be made public (by site admins), which means
all users can view them. This is a great way to share useful search
queries with all users of a Sourcegraph instance.
- Saved searches can be marked as a "draft", which is a gentle indicator
that other people shouldn't use it yet.
I noticed that the commit page doesn't render well on mobile when the
commit has a commit message.
This commit refactors how the `Commit` component is rendered, including
on mobile, which affects both the commit**s** and the commit page.
The two most important changes:
- The component now uses CSS grid to be more flexible about how
individual elements are arranged.
- On mobile we don't show expand the message inline anymore but instead
show it full screen. I think that works well for the commits list too
because now you can open and read a longer commit message without having
to scroll the commits list itself.
## Test plan
Manually inspecting the commits and commit pages. Opened a long commit
message to test that the message is properly scrollable.
<!-- PR description tips:
https://www.notion.so/sourcegraph/Write-a-good-pull-request-description-610a7fd3e613496eb76f450db5a49b6e
-->
Opsgenie alert notifications for critical alerts should be enabled by
default for production projects or where `env.alerting.opsgenie` is set
to true.
Closes CORE-223
## Test plan
Tested locally by running `sg msp gen` for a `prod` env which doesn't
have an alerting config and verifying that notification suppression was
disabled
Set `env.alerting.opsgenie` to false which enabled suppression again.
No changes to `test` environments unless `env.alerting.opsgenie` is set
to true.
This PR modifies the sub_repo_permissions database to store the ip
addresses associated with each Perforce path rule, as part of the IP
based sub repo permissions work.
The new IP column is implemetned as a []text similar to the path column.
The IP address associated with `paths[0]` is stored in the ips column in
`ips[0]`.
For example, the follownig proections table
```
Protections:
read user emily * //depot/elm_proj/...
write group devgrp * //...
write user * 192.168.41.0/24 -//...
write user * [2001:db8:1:2::]/64 -//...
write user joe * -//...
write user lisag * -//depot/...
write user lisag * //depot/doc/...
super user edk * //...
```
turns into the following rows in the sub_repo_permissions table
| repo_id | user_id | version | updated_at | paths | ips |
|---------|---------|---------|------------|-------|-----|
| 1 | 1 | 1 | 2023-07-01 10:00:00 | {`"//depot/elm_proj/..."`} | {`"*"`}
|
| 1 | 2 | 1 | 2023-07-01 10:00:00 | {`"//..."`} | {`"*"`} |
| 1 | 3 | 1 | 2023-07-01 10:00:00 | {`"-//..."`} | {`"192.168.41.0/24"`}
|
| 1 | 4 | 1 | 2023-07-01 10:00:00 | {`"-//..."`} |
{`"2001:db8:1:2::]/64"`} |
| 1 | 5 | 1 | 2023-07-01 10:00:00 | {`"-//..."`} | {`"*"`} |
| 1 | 6 | 1 | 2023-07-01 10:00:00 | {`"-//depot/..."`,
`"//depot/doc/..."`} | {`"*"`, `"*"`} |
| 1 | 7 | 1 | 2023-07-01 10:00:00 | {`"//..."`} | {`"*"`} |
## Test plan
The unit test for the sub_repository_permissions store PR that is built
on this PR:
https://app.graphite.dev/github/pr/sourcegraph/sourcegraph/63811/internal-database-sub_repo_permissions-modify-store-to-be-able-to-insert-ip-based-permissions
## Changelog
- The sub_repo_permissions table now has an ips column to store the
associated IP address associated with each path rule.
The branches page didn't work well on mobile and neither did the tags
page if long tag names were present.
This commit changes how the information is displayed and rendered,
especially on mobile.
I also added additional links to each row to make navigating to relevant
places easier.
## Test plan
Manual inspection of pages in various screen sizes.
This improves the typings to remove some inscrutable inferred types and
some weird type errors if you didn't use React.PropsWithChildren. Also
refactors the code and exposes a new component AuthenticatedUserOnly
that's simpler and can be used when you don't need to do prop
propagation of authenticatedUser.
## Test plan
CI
Contributes to SRCH-738
Notably, this does not yet identify the root cause of SRCH-738, but it
does identify and fix some confounding bugs. It's possible that these
actually also _cause_ some of the issues in SRCH-738, but I wanted to at
least push these to dotcom, where we can reproduce some of the
weirdness. At the very least, it doesn't explain the auth errors being
reported.
The goal of this PR is to increase the stability of web-sveltekit
e2e-tests so that we don't have to rely on manual runs anymore. They
have previously been disabled due to a high number of failures:
https://github.com/sourcegraph/sourcegraph/pull/63874
---
To improve the stability of web-sveltekit e2e-tests, I used `sg ci bazel
test //client/web-sveltekit:e2e_test --runs_per_test=N` with N=5,10,15
to see which tests break under different levels of pressure on the
machine. The logs looked like it was mostly timeouts, that got worse
when increasing N. That means we can check where tests will break due to
timeouts, but we don't really need to raise timeouts so far that it
would work with N=20.
With N=5, 10 we get a good understanding if our timeouts are high
enough.
You can see two CI runs here after applying higher timeouts and skipping
a consistently failing test:
- N=5:
https://buildkite.com/sourcegraph/sourcegraph/builds/283011/waterfall
- N=10:
https://buildkite.com/sourcegraph/sourcegraph/builds/283013/waterfall
---
From logs of some other run that I don't have the link to anymore, we
can see that some tests take up to 50s so a timeout of 60s (instead of
the default 30s) for CI should be a good new ceiling.
```
Slow test file: [chromium] › src/routes/[...repo=reporev]/(validrev)/(code)/-/blob/[...path]/page.spec.ts (48.9s)
--
| Slow test file: [chromium] › src/routes/[...repo=reporev]/(validrev)/(code)/page.spec.ts (45.0s)
| Slow test file: [chromium] › src/routes/search/page.spec.ts (40.6s)
| Slow test file: [chromium] › src/routes/[...repo=reporev]/(validrev)/(code)/-/tree/[...path]/page.spec.ts (31.7s)
| Slow test file: [chromium] › src/routes/layout.spec.ts (31.5s)
```
## Test plan
CI
## Changelog
<!-- OPTIONAL; info at
https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c
-->
It turns out that Kubernetes objects constructed using client-go don't
know their own TypeMeta. There is code in client-go that figures out
which resource-scoped HTTP path to call on the kube-apiserver by looking
up a mapping of Go object kinds to k8s kinds. A similar facility appears
to be exposed to the user by apiutil.GVKForObject().
Closes
https://linear.app/sourcegraph/issue/REL-275/reconciler-logs-dont-contain-resource-metadata
Secrets fetched from GSM should probably not be stored locally. As we
increase the usage of fetching external secrets, this stuff is
increasingly sensitive, particularly for SAMS stuff - every time it's
used, we should ensure that the user has the required permissions, and
also only store external secrets in-memory.
It looks like several other callsites make use of the persistence of
other secrets e.g. those prompted from users, so this change
specifically targets the `GetExternal` method. Additionally, I also
added a check on load to delete any legacy external secrets that are
stored to disk on load - we can remove this after a few weeks.
## Test plan
Unit tests asserts old behaviour and new desired behaviour
`sg start -cmd cody-gateway` uses external secrets and works as expected
After running `sg`, `sg secret list` has no external secrets anymore
Closes
[DINF-58](https://linear.app/sourcegraph/issue/DINF-58/overwrite-ordering-in-sg)
https://github.com/user-attachments/assets/d8e59a5f-9390-47f7-a6a7-9ccbf97423f8
## Test plan
- Add a `commandset` to the `sg.config.overwrite.yaml`
- This commandset should depend on an existing command in the
`sg.config.yaml` file.
- The commandset should also include an `env var` that should override
what's set in the `command` contained in the `sg.config.yaml` file.
- Running `sg start <commandset name>` should allow the env ordering
matrix shown below
```
Priority: overwrite.command.env > overwrite.commandset.env > command.env > commandset.env.
```
## Changelog
N/A
Delivery Manifest step has started to run `bazel build` commands, in them clobbering our execlog artifacts. We should only emit it for the test buildkite jobs (at least for the time being), as it currently doesnt make sense for e.g. the image push jobs which contain multiple invocations
## Test plan
CI
## Changelog
Follow-up to
https://sourcegraph.slack.com/archives/C07KZF47K/p1720639713491779?thread_ts=1720636753.404169&cid=C07KZF47K
Basically, if we see that the local assets are above 500mb, we just nuke
it. It's a bandage btw. The `.gitkeep` is there so it doesn't break the
build because there's nothing to embed.
@eseliger and @burmudar can you test this a bit further and land it if
it's all good? My tests are good, but I don't want to hastily land
something and go in PTO five seconds before I'm out for two weeks.
## Test plan
Locally tested.
---------
Co-authored-by: William Bezuidenhout <william.bezuidenhout@sourcegraph.com>
Fixes CODY-2888
Previously, Sourcegraph Enterprise instances with context filters
enabled rejected requests from all unknown clients out of concern that
they might not respect context filters. This behavior makes it
incredibly impractical to release now agent-based clients (CLI, Eclipse,
Visual Studio, Neovim, ..) that do respect context filters out of the
box thanks to the reused logic in the Cody agent.
This logic suffers from both false positives and false negatives:
- False negatives: upcoming Cody clients (CLI, Eclipse, Visual Studio)
already support context filters out of the box thanks to using the Cody
agent but they can't send requests unless we add a special case to them.
It may require months for these clients to wait for all Enterprise
instances to upgrade to a version that adds exceptions for their name.
- False positive: a malicious client can always fake that it's
"jetbrains" with a valid version number even if the client doesn't
respect context filters. This gives a false sense of security because it
doesn't prevent malicious traffic from bypassing context filters. In
fact, I am leaning towards using the
Now, with this change, Sourcegraph Enterprise instances only reject
requests from old versions of Cody clients that are known to not support
context filters. This ensures we never have false positives or false
negatives.
<!-- PR description tips:
https://www.notion.so/sourcegraph/Write-a-good-pull-request-description-610a7fd3e613496eb76f450db5a49b6e
-->
## Test plan
See updated test case which accepts a request from an unknown "sublime"
client.
<!-- REQUIRED; info at
https://docs-legacy.sourcegraph.com/dev/background-information/testing_principles
-->
## Changelog
<!-- OPTIONAL; info at
https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c
-->
Instead of a a maximum of 2 minor versions beyond the deployed
Sourcegraph instance.
The main advantage of removing the 2-minor-version constraint, is that
it allows admins to always be able to update to the latest SG with one
click, instead of having to repeat that process through intermediate
versions if they have fallen far behind.
Otherwise Apollo Client complains that it does not know how to cache the
result because it does not know which repository ID to associate the
result with.
## Test plan
CI
We previously improved the performance of Language Stats Insights by
introducing parallel requests to gitserver:
https://github.com/sourcegraph/sourcegraph/pull/62011
This PR replaces the previous approach where we would iterate through
and request each file from gitserver with an approach where we request
just one archive. This eliminates a lot of network traffic, and gives us
an additional(!) performance improvement of 70-90%.
Even repositories like chromium (42GB) can now be processed (on my
machine in just one minute).
---
Caching: We dropped most of the caching, and kept only the top-level
caching (repo@commit). This means that we only need to compute the
language stats once per commit, and subsequent users/requests can see
the cached data. We dropped the file/directory level caching, because
(1) the code to do that got very complex and (2) we can assume that most
repositories are able to compute within the 5 minutes timeout (which can
be increase via the environment variable `GET_INVENTORY_TIMEOUT`). The
timeout is not bound to the user's request anymore. Before, the frontend
would request the stats up to three times to let the computation move
forward and pick up where the last request aborted. While we still have
this frontend retry mechanism, we don't have to worry about an
abort-and-continue mechanism in the backend.
---
Credits for the code to @eseliger:
https://github.com/sourcegraph/sourcegraph/issues/62019#issuecomment-2119278481
I've taken the diff, and updated the caching methods to allow for more
advanced use cases should we decide to introduce more caching. We can
take that out again if the current caching is sufficient.
Todos:
- [x] Check if CI passes, manual testing seems to be fine
- [x] Verify that insights are cached at the top level
---
Test data:
- sourcegraph/sourcegraph: 9.07s (main) -> 1.44s (current): 74% better
- facebook/react: 17.52s (main) -> 0.87s (current): 95% better
- godotengine/godot: 28.92s (main) -> 1.98s (current): 93% better
- chromium/chromium: ~1 minute: 100% better, because it didn't compute
before
## Changelog
- Language stats queries now request one archive from gitserver instead
of individual file requests. This leads to a huge performance
improvement. Even extra large repositories like chromium are now able to
compute within one minute. Previously they timed out.
## Test plan
- New unit tests
- Plenty of manual testing
Precursor for
https://linear.app/sourcegraph/issue/GRAPH-735/test-syntactic-usages
This PR introduces the `MappedIndex` abstraction, which wraps up an
upload with a target commit. Its APIs then take care of mapping upload
relative paths and repo relative paths, and ranges across commits.
My main motivation for making this change is that I can now test this
logic in isolation (which this PR does), and also have an interface that
is easy to fake and use to test syntactic usages.
## Test plan
Added unit tests for the `MappedIndex` component, manual testing of the
GraphQL APIs show no changes in the syntactic usages output between this
PR and master.
Turns out, `gorm` does not auto-migrate constraints you've removed from
the struct schema. You must drop them by hand, or run the commands to
drop them explicitly.
## Test plan
```
sg start -cmd enterprise-portal
```
```
\d+ enterprise_portal_*
```
then, comment the custom migrations, and run `sg start -cmd
enterprise-portal` again to re-do the migrations. The output of `\d+`
matches. Unexpected constraints are gone.
This commit adds the Svelte version of the compare page. Now that we
have a revision picker this was easy to implement.
A couple of notes:
- The URL parameter is a rest parameter ([...spec]) because revisions
can contain slashes, and we don't encode those.
- I changed the API of the revision selector so that it works on pages
that don't have `resolvedRevision` available (also we never needed the
`repo` property from that object), and to provide custom select
handlers.
- Moved the revision to lib since it's used on various different pages.
- Unlike the React version this version
- Provides forward/backward buttons to browse the commit list
- Uses infinity scrolling to load the next diffs
- Doesn't use a different style for the commit list. I experimented with
adding a "compact" version of the commit but it didn't feel quite right.
I'm sure we'll eventually redesign this.
- The `filePath` parameter is supported (for backwards compatibility)
but we don't use it anywhere atm.
(note that the avatar size has changed since the video was recorded;
it's larger now)
## Test plan
Manual testing.
We recently updated the completions APIs to use the `modelconfig` system
for managing LLM model configuration. Behind the scenes, we
automatically converted the existing site configuration ("completions
config") into the newer format so things work as expected.
However, the GraphQL view of the Sourcegraph instance's LLM
configuration was not updated to use the `modelconfig` system. And so fi
the site admin and opted into using the new-style of configuration data,
the data returned would be all sorts of wrong.
(Because the GraphQL handler looked for the "completions config" part of
the site config, and not the newer "model configuration" section.)
This PR updates the `CodyLLMConfiguration` GraphQL resolver to return
the data from the modelconfig component of the Sourcegraph instance.
Some careful refactoring was needed to avoid a circular dependency in
the Go code. So the resolver's type _declaration_ is in the
`graphqlbackend` package. But it's _definition_ is in
`internal/modelconfig`.
## Test plan
I only tested these changes manually.
If you open the Sourcegraph instance's API console, this is the GraphQL
query to serve all of the data:
```gql
{
site {
codyLLMConfiguration {
chatModel
fastChatModel
completionModel
provider
disableClientConfigAPI
chatModelMaxTokens
fastChatModelMaxTokens
completionModelMaxTokens
}
}
}
```
## Changelog
NA
Part of: https://github.com/sourcegraph/devx-support/issues/1093
If we get 3 errors in a row trying to write to bigquery ... chances are
we are not going to succeed. So we exit early.
## Test plan
CI
## Changelog
- sg: provide a better error message when we fail to insert into
bigquery
- sg: stop puslishing to bigquery if we get 3 errors in a row
Testing for display-name setting which we recently added, and this is
useful in the interim to set display names on the go for subscriptions
EP already tracks.
note: I don't anticipate doing this for every field we make update-able,
especially since the next step(s) will be updating the UI
## Test plan
```
sg enterprise subscription set-name es_4dae04ba-5f5b-431a-b90b-e8e3dd449181 "robert's test subscription"
```
Adds site-config configuration for RFC 969 intent detection, making the
Intent Detection API endpoint and token configurable without code
changes. Additionally, adds an option to hit multiple intent detection
backends with the same query.
Previously, URL was hardcoded in code, so if the backend has changed, we
had to redeploy sourcegraph.com.
As we iterate on intent detection, we want to be able to test multiple
models in parallel, so this PR adds a setting for `extra` backends - if
provided, additional .com -> backend requests will be sent, but the
client-initiated request will not wait for those requests.
Closes AI-128.
## Test plan
- tested locally - add
```
"cody.serverSideContext": {
"intentDetectionAPI": {
"default": {
"url": "http://35.188.42.13:8000/predict/linearv2"
},
"extra": [
{
"url": "http://35.188.42.13:8000/predict/linearv2"
}
]
}
}
```
to `experimentalFeatures` in dev-private.
This is an input element that constraints the input to look like a path
component, used for the names of batch changes (and for prompts in the
prompt library in the future).
Also fixes an invalid regexp (it needs to escape `-` because the `'v'`
flag is used; see
https://developer.mozilla.org/en-US/docs/Web/HTML/Attributes/pattern).
## Test plan
Test against existing batch change create form name input. Try invalid
inputs.
This was added to the old global navbar but not the one one by accident
(by me).
## Test plan
Ensure that the new global navbar shows a Saved Searches link.
Ideally, we would introduce copies of the old names, deprecate the old
ones, migrate the frontend code, and cut over after a couple of minor
releases.
I don't have time for that now, so just updating the docs a bit for clarity.
Just a couple of things I noticed while working on unrelated tasks.
1) Removes the unused `Separator` component, which has been replaced by
the Panel API
2) Makes the sourcegraph mark a proper icon so it can be used via the
`Icon` component like all our other icons.
While attempting to use `observeIntersection` with the new references
panel, I was running into performance issues with large lists of
references (the API does not actually paginate yet). When I took a look
at the profile, a good chunk of the time came from finding the nearest
scrolling ancestor, specifically the `overflowY` part, which requires
computing large chunks of the layout.
So this changes the `observeIntersection` action to take an explicit
target container so we don't need to do any searching for a scroll
container. For the example that I was debugging, this reduced the time
it took to render the list ~5x. Additionally, it establishes a cache for
all created `IntersectionObserver`s rather than just the root observer.
S2 Cody Web is broken at the moment. New client-config handlers fail
with 401 status because we don't send custom headers, this works for gql
queries since they all are POST requests and the browser automatically
sends an Origin header for them and this is enough for our auth
middleware to check cookies, but with client-config which is rest it's
not the case and we should send `X-Requested-Client: Sourcegraph` header
to make our auth middleware to pass this query correctly
Note that this problem doesn't exist in local builds since we proxy all
requests and add `X-Requested-Client: Sourcegraph` in dev server.
See Cody latest build PR for more details
https://github.com/sourcegraph/cody/pull/4898
## Test plan
- CI is passing
`sg run` is supposed to be deprecated in favour of `sg start -cmd`, but
the `sg start` completions don't work with `-cmd` like `sg run` does.
This change updates `sg start` completion to check for the `-cmd` flag,
and if it is provided, offer completions for commands instead of
command_sets_ (the default `sg start` behaviour).
## Test plan
<img width="1023" alt="image"
src="https://github.com/user-attachments/assets/9b887180-f58f-4aef-9dbb-718c71ba15e6">
<img width="1077" alt="image"
src="https://github.com/user-attachments/assets/927b4562-fce1-48c0-a8c5-453bfc60fe35">
## Changelog
- Completions for `sg start -cmd` now offer valid suggestions.
Noticed several `Usage` using newlines, which makes `-h` output pretty
annoying to read as it breaks up the formatting. It tickled me enough to
put a formatting check against it, and update the existing usages that
were incorrect, to use `Description` or `UsageText` instead :-)
## Test plan
CI, `sg -h` is pretty(er) again (but still very long)
- Don't show the Tools menu on dotcom for now, since there is only 1
item. Users can still access saved searches in their user menu.
- Fix an issue where non-site admins on dotcom would see an error at
`/saved-searches` until they changed the Owner filter.
- Other minor code cleanups.
## Test plan
Try being a non-site admin and ensure that `/saved-searches` works.
The appliance checks owner references, and only deletes (or updates)
resources whose owner references matches the configmap being reconciled.
The current user interface to external databases, is for the admin to
create secrets with a well-known name out of band (e.g. "pgsql-auth")
and then disable the relevant backing services (e.g. pgsql). This commit
fixes a bug in which the appliance would have deleted such secrets,
leaving the admin unable to use external databases.
Discovered while investigating
https://linear.app/sourcegraph/issue/REL-14/ensure-pods-roll-when-referenced-secret-changes,
but does not close it.
