Automatically generated PR to update package lockfiles for Sourcegraph
base images.
Built from Buildkite run
[#283970](https://buildkite.com/sourcegraph/sourcegraph/builds/283970).
## Test Plan
- CI build verifies image functionality
Co-authored-by: Buildkite <buildkite@sourcegraph.com>
As part of the [Vuln Scanning
Improvements](https://linear.app/sourcegraph/project/[p0]-vulnerability-scanning-improvements-75299c4312dd/issues)
project, I've been working on tooling to automate the security
approval step of the release process.
This PR integrates these improvements into the release pipeline:
* Internal releases will run a vulnerability scan
* Promote-to-public releases will check for security approval
If a public release does not have security approval, it will block the
promotion process. The step happens at the start of the pipeline so
should be a fast-fail. You can also check for release approval before
running promotion by running `@secbot cve approve-release
<version>` in the #secbot-commands channel. In an ideal world we
(security) will have already gone through and approved ahead of release.
I've tested this PR as much as I can without running an actual
release! We have a 5.5.x release tomorrow so it'll be a good test.
If it does cause problems that can't be easily solved, it can always
be temporarily disabled.
I've tagged this PR to be backported to `5.5.x`.
## Pre-merge checklist
- [x] Revert commit that disables release promotion
## Test plan
Manual testing of the release process:
- [x] [Successful test
run](https://buildkite.com/sourcegraph/sourcegraph/builds/283774#0190dfd6-fa70-4cea-9711-f5b8493c7714)
that shows the security scan being triggered
- [x] [Promote to public test
run](https://buildkite.com/sourcegraph/sourcegraph/builds/283826) that
shows the security approval approving a release
- [x] [Promote to public test
run](https://buildkite.com/sourcegraph/sourcegraph/builds/283817#0190e0ec-0641-4451-b7c7-171e664a3127)
that shows the security approval rejecting a release with un-accepted
CVEs
## Changelog
<br> Backport 9dd901f3c9 from #63990
Co-authored-by: Will Dollman <will.dollman@sourcegraph.com>
In order to run nightly vulnerability scans of Sourcegraph releases, we
need to publish a new set of images whenever the release branch is
pushed to.
Previously, this was implemented in
https://github.com/sourcegraph/sourcegraph/pull/63379 but with RFC 795
the release branch format changed from 5.5.1234 to 5.5.x.
This PR updates the regex to catch this new format.
The end result of this is that whenever Buildkite runs on a branch
matching `\d.\d.x`, it will push images to the
`us.gcr.io/sourcegraph-dev/gitserver` registry with the tag
`$branch-insiders`.
I've also tagged this PR for backport as we want it on the current
patch release branch 5.5.x :)
## Test plan
- Test buildkite run on branch `will-0.0.x` (with modified regex to
match that branch)
https://buildkite.com/sourcegraph/sourcegraph/builds/283608
## Changelog
<br> Backport b7242d280f from #63985
Co-authored-by: Will Dollman <will.dollman@sourcegraph.com>
Currently events are triggered whenever a user signs in with
`http-header` auth. This is because of the `GetAndSaveUser` function
always triggering an event.
However, before the new telemetry events, these events were only created
when a new user was created.
This PR brings the new telemetry code in line with the old telemetry
code to stop the massive amounts of spam caused by this event.
Closes SRC-461
## Test plan
Adjust expected events in unit test.
## Changelog
- Fixed an issue where the `http-header` auth would cause a massive
amount of event logs spam
<br> Backport cd65951961 from #63843
Co-authored-by: Petri-Johan Last <petri.last@sourcegraph.com>
Contributes to SRCH-738
Notably, this does not yet identify the root cause of SRCH-738, but it
does identify and fix some confounding bugs. It's possible that these
actually also _cause_ some of the issues in SRCH-738, but I wanted to at
least push these to dotcom, where we can reproduce some of the
weirdness. At the very least, it doesn't explain the auth errors being
reported.
(cherry picked from commit d91fab39e2)
Co-authored-by: Michael Bahr <michael.bahr@sourcegraph.com>
backport https://github.com/sourcegraph/sourcegraph/pull/63863
S2 Cody Web is broken at the moment. New client-config handlers fail
with 401 status because we don't send custom headers, this works for gql
queries since they all are POST requests and the browser automatically
sends an Origin header for them and this is enough for our auth
middleware to check cookies, but with client-config which is rest it's
not the case and we should send `X-Requested-Client: Sourcegraph` header
to make our auth middleware to pass this query correctly
Note that this problem doesn't exist in local builds since we proxy all
requests and add `X-Requested-Client: Sourcegraph` in dev server.
See Cody latest build PR for more details
https://github.com/sourcegraph/cody/pull/4898
## Test plan
CI
Co-authored-by: Vova Kulikov <vovakulikov@icloud.com>
Closes SRCH-723
The baseURL for GitHub apps defaults to `https://github.com` when no
`externalServiceURL`, we somehow missed this during our testing.
## Test plan
Manual testing with the GHE instance.
## Changelog
<br> Backport 1c40c9e5bc from #63803
Co-authored-by: Bolaji Olajide <25608335+BolajiOlajide@users.noreply.github.com>
Co-authored-by: Anish Lakhwara <anish+github@lakhwara.com>
See https://github.com/sourcegraph/sourcegraph/pull/63870
cc @sourcegraph/release
## Test plan
Covered by existing tests
## Changelog
- Adds an experimental feature `commitGraphUpdates` to control how
upload visibility is calculated.
This PR upgrades the cody web experimental package to 0.2.5, in the new
version we fixed
- Telemetry problem with init extension-related events (we don't
send install extension events anymore)
- Most recent updates on LLM availability for enterprise instances
## Test plan
- CI is green
- Manual check on basic Cody Web functionality (highly recommended) <br>
Backport e6bd85e4b7 from #63839
Co-authored-by: Vova Kulikov <vovakulikov@icloud.com>
This PR fixes an important bug in #62976, where we didn't properly
map the
symbol line match to the return type. Instead, we accidentally treated
symbol
matches like file matches and returned the start of the file.
## Test plan
Add new unit test for symbol match conversion. Extensive manual testing.
<br> Backport 004eb0fd83 from #63773
Co-authored-by: Julie Tibshirani <julietibs@apache.org>
The OTEL upgrade https://github.com/sourcegraph/sourcegraph/pull/63171
bumps the `prometheus/common` package too far via transitive deps,
causing us to generate configuration for alertmanager that altertmanager
doesn't accept, at least until the alertmanager project cuts a new
release with a newer version of `promethues/common`.
For now we forcibly downgrade with a replace. Everything still builds,
so we should be good to go.
## Test plan
`sg start` and `sg run prometheus`. On `main`, editing
`observability.alerts` will cause Alertmanager to refuse to accept the
generated configuration. With this patch, all is well it seems - config
changes go through as expected. This is a similar test plan for
https://github.com/sourcegraph/sourcegraph/pull/63329
## Changelog
- Fix Prometheus Alertmanager configuration failing to apply
`observability.alerts` from site config <br> Backport
ffa873f3ad from #63790
Co-authored-by: Robert Lin <robert@bobheadxi.dev>
This will correct6 upgrade path for mvu plan creation
## Test plan
CI test
## Changelog
<br> Backport cb19d6f0a9 from #63764
Co-authored-by: Warren Gifford <warren@sourcegraph.com>
Missing bit for the minor release version bump
## Test plan
CI
<br> Backport 087ad83995 from #63767
Co-authored-by: Jean-Hadrien Chabran <jean-hadrien.chabran@sourcegraph.com>
We created a decoder that was never used, but the package is otherwise
unused. It recently had a CVE, so this just removes it so it's no longer
part of our security surface area.
Make SetupEnvtest slightly lower-level by asking callers to construct
their own client from the returned k8s REST config. This is because
there are 2 kinds of official kubernetes client in Go - a
kubernetes.Clientset and a client.Client. The latter is
more-traditionally used in operators, because it's what a
ControllerManager.GetClient() returns, and indeed this is what our
reconciler uses.
We ended up using a kubernetes.Clientset in the envtest-using golden
tests for the reconciler, because its mechanics for listing resources
were simpler. Now, I want to reuse SetupEnvtest somewhere that needs a
client.Client. We could undertake work to use only one flavor of
kubernetes client, but this commit seems like a decent low-cost first
step.
Docker images executor, executor-kubernetes, bundled-executor has
reported high/critical CVE-2024-24790 , CVE-2023-45288 reported on
golang stdlib. Upon testing, src version 5.3.0 was using `1.20.x` as per
e8e79e0311
This pull request attempts to upgrade src version to 5.4.0
## Test plan
- CI 🟢
- src version should report 5.4.0 (I built the image locally and tested
it)
`docker run --platform linux/amd64 -it --entrypoint /bin/sh
executor:candidate`
## Changelog
<!-- OPTIONAL; info at
https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c
-->
Upgrade src-cli version to 5.4.0 to address CVE-2024-24790 ,
CVE-2023-45288
Currently if a cloud ephemeral build is trigger it is triggered on the
`main` sourcegraph pipeline. Once a build a triggered and a commit is
subsequently pushed the previous build is cancelled - which means the
Cloud Ephemeral build is cancelled leading to a failed deployment.
In this PR, we instead trigger a build on the Cloud Ephemeral pipeline.
Which is the _exact_ pipeline as `sourcegraph` main but:
- sets the pipeline env to always have `CLOUD_EPHEMERAL=true`
- does not cancel previous builds
## Test plan
https://buildkite.com/sourcegraph/cloud-ephemeral/builds/1
## Changelog
* `sg cloud eph` will now trigger builds on the `cloud-ephemeral`
pipeline
Automatically generated PR to update package lockfiles for Sourcegraph
base images.
Built from Buildkite run
[#281769](https://buildkite.com/sourcegraph/sourcegraph/builds/281769).
## Test Plan
- CI build verifies image functionality
Co-authored-by: Buildkite <buildkite@sourcegraph.com>
Since we removed on-demand cloning, the scheduler is now expected to always contain all repositories. Thus, we no longer need to constrain the set of uncloned repos to a ginormous ID list.
Test plan:
CI still passes.
This syncer doesn't depend on anything in repo updater, so we're moving it to worker instead, where it can selectively be disabled and is properly monitored.
Test plan:
CI passes, code review.
We have been using v2 data since >5 years now, this should be safe to
remove.
As a side-effect, we have one less background task running in frontend,
which means it ran N times in horizontally scaled environments, which
isn't exactly useful.
Test plan:
Code review.
Currently, nothing really tells that Cody Gateway needs redis, the env
var for finding the address is hidden somewhere deep in the redispool
package.
In practice, we only use one redis instance, but at some point we
started using both redispool.Cache and redispool.Store, which means we
maintain two connection pools, leading to more than expected
connections.
Test plan:
Code review and CI.
This PR restructures the packages to move all symbols-only code into the
symbols service. This helps to reason better about which service is
accessing what datastores.
Test plan:
Just moved code, compiler and CI are happy.
Recently, this was refactored to also allow using the redispool.Store.
However, that makes it very implicit to know where something is being
written, so instead we pass down the pool instance at instantiation.
This also gives a slightly better overview of where redispool is
actually required.
Test plan: CI passes.
This PR has a single goal: pass a new `ModelConfigInfo` type, which has
the `Provider` and `Model` we should use to serve a completions request,
down into `client.Get()`
This PR explicitly only handles the case we care about for Self-hosted
models, and all pieces of logic that I expect will be
replaced/superseded by your work @chrsmith are annotated with `//
TODO(slimsag): self-hosted-models:` comments so we can easily find and
remove them when your work is ready.
Every location I have modified has been carefully wrapped in an if
statement like `if conf.Get().SiteConfig().ModelConfiguration != nil` to
ensure that this change _only_ affects people who set the new
`"modelConfiguration"` site config property.
## Test plan
1. Configured `"modelConfiguration"` and removed `"completions"` in my
dev instances' site config.
2. Used VS Code chat to confirm I see this codepath is connected
end-to-end and I get a `TODO` error back:
<img width="679" alt="image"
src="https://github.com/sourcegraph/sourcegraph/assets/3173176/d3c07ffe-7ec2-4ad9-9251-2272ce36d44b">
## Changelog
N/A
---------
Signed-off-by: Stephen Gutekanst <stephen@sourcegraph.com>
Closes srch-103
Currently we don't show the global context filter on the search home
page or the search results page (global context is the default context).
This commit does two things:
- It prefills the search input on the search homepage with the user's
default context (like the React app)
- It the logic that pre-processed the search query and removed the
context filter if it was global.
In other words we simplify the query logic by showing/submitting the
search query as is. Notably this doesn't affect how the search input
works on repo pages.
## Test plan
- Opening the search home page pre-fills the query input with the
default search context
- Submitting a query without a `context:` filter does not add a
`context:` filter to the URL or the search input
- If a query contains `context:global` that filter is preserved in the
query input (it wasn't before)
We don't like the style/alignment of the Lucide case sensitivity and
regex icons (the structural search icon seems fine). This commit
switches them back to the MDI icons that we used before.
Solution or temporary workaround (tbd) for srch-544
## Test plan
Manual testing.
Implements upsert for all the subscriptions fields in the DB client. As
part of this I generalized the logic for building upsert DB interactions
into a new `upsert` package, because this pattern is a common one we'll
need to implement to maintain various AIP-update-compliant endpoints,
which specifies various upsert behaviours: https://google.aip.dev/134
Part of CORE-216
Part of CORE-156
## Test plan
Integration tests against DB
This PR bundles several small refactoring to make it easier to review
the pending "refactor completion APIs to read from modelconfig".
Going commit-by-commit:
[Export the ValidateModelRef
function](3c949a88d0)
This just exports a validation function that was previously private from
the `internal/modelconfig` package. We'll be using this to sanity check
the `ModelRef` that is returned from the "getModelsFn".
[Add modelconfig.{InitMock,
ResetMock}](f807f40bf5)
In order to update the `frontend/internal/httpapi/completions` unit
tests, we need to have changes to the site configuration data _ALSO_
update the global `modelconfig.Service`. Typically this would happen by
a "config watcher" that gets registered, but we don't do that as part of
unit tests.
So instead, I just added some clunky `InitMock() error` and `ResetMock()
error` functions to support unit testing. If you have a better idea for
how to expose this behavior, I'm all ears.
[Move fireworks-specific API types into
types.go](bb02ccba45)
Moves the API data types for the `fireworks` client into their own file,
just to keep things tidy.
[Refactor completion
clients](02a5e3bfc9)
This is a bit involved, but from a high-level, we are just passing the
`types.CompletionRequest` object "lower" into the call stack. Rather
than only return the `.Parameters` field
(`types.CompletionRequestParameters`).
This is necessary, because I plan on adding another field to
`types.CompletionRequest` (to include the `modelconfig.Provider` and
`modelconfig.Model` data). So this change just does some of the work for
that now, resulting in a smaller and easier to review diff later.
Also, in some cases I moved the calls to
`tokenManager.UpdateTokenCountsFromModelUsage` into a dedicated function
to simplify the callsites.
## Test plan
Existing unit tests
## Changelog
NA
