Closes https://github.com/sourcegraph/sourcegraph/issues/54829
Next steps:
- what safeguards will we have in place for when we remove this from `sg lint`
- should we remove it from `sg lint`, its likely not contributing much overhead even in the uncommon case of shell files being touched
_How did some of these violations get in when theres shell lints :clueless:_
## Test plan
`pre-commit run --hook-stage=pre-push shellcheck shfmt`
Relates to #59329
This is mostly a refactor to prepare for a JSON writer that replaces our CSV writer.
The high-level plan is to switch the internal (object storage) AND the external (download) format from CSV to JSON.
Test plan:
- CI
- Note: I had to update `SearcherFake`, hence some of the test data changed. It only affects a few tests and the character of the tests remains the same.
We sometimes pass huge data here, and this can cause a very large attribute to be rendered. This PR limits it to at most 128 characters, which was chosen arbitrarily.
https://github.com/sourcegraph/sourcegraph/pull/59284 dramatically reduced the `mean_blocked_seconds_per_conn_request` issues we've been seeing, but overall delays are still higher, even with generally healthy Cloud SQL resource utilization.
<img width="1630" alt="image" src="https://github.com/sourcegraph/sourcegraph/assets/23356519/91615471-5187-4d15-83e7-5cc94595303c">
Spot-checking the spikes in load in Cloud SQL, it seems that there is a variety of causes for each spike (analytics workloads, Cody Gateway syncs, code intel workloads, gitserver things, `ListSourcegraphDotComIndexableRepos` etc) so I'm chalking this up to "expected". Since this alert is seen firing on a Cloud instance, let's just relax it for now so that it only fires a critical alert on very significant delays.
* close open access requests when user is created
* update test
* fix test
* remove empty lines
* remove verified email check
* update test
* Minor PR feedback (#59416)
---------
Co-authored-by: Petri-Johan Last <petri.last@sourcegraph.com>
* adjust logic and replace 'show all' with the show 'more/less' button
* ensure logic works when searching for filters
* remove state for allFiltersDisplayed to a static value
* Reset usage cache automatically on interval change
* address feedback
* Comments, Cleanups, Tests for `naman/reset-cody-usage-cache` (#59453)
* Add limiter.RecordingRedisStoreFake for tests
* Comments and Cleanups
* Add unit test for usage cache expiration
* Add new Bazel build dependency
* Run 'bazel run //:configure'
* Update cmd/cody-gateway/internal/actor/actor.go
Co-authored-by: Robert Lin <robert@bobheadxi.dev>
* Update cmd/cody-gateway/internal/actor/actor.go
Co-authored-by: Robert Lin <robert@bobheadxi.dev>
---------
Co-authored-by: Robert Lin <robert@bobheadxi.dev>
* Address TTL > interval logic placement feedback
---------
Co-authored-by: Chris Smith <chrsmith@users.noreply.github.com>
Co-authored-by: Robert Lin <robert@bobheadxi.dev>
This bump only includes one commit from zoekt which is a performance
fix. See the commit in the zoekt repo for details.
- 7487a0d531 chunkmatches: reuse last calculated column when filling
Test Plan: tested in zoekt repo and this CI
* logging configs during siteconfig changes
* logging previous quota values
* changing logging
* adding more information to logging
* logging previous and new codehost config
* adding email to verification log
* Update internal/auth/userpasswd/reset_password.go
Co-authored-by: Vincent <evict@users.noreply.github.com>
* feedback changes
* variable fix
* removing prev quota logging due to extra db call
* spacing
---------
Co-authored-by: Vincent <evict@users.noreply.github.com>
This change implements the searcher logic for 'and' and 'or' queries:
* Renames `matcher` to `matchTree` to be more consistent with Zoekt and
gitserver
* Adds `andMatchTree` and `orMatchTree`, plus a bunch of tests
This code is not yet hooked in end-to-end. In a final follow-up PR, we'll
update the jobs code to pass entire basic queries to searcher, and translate
them to these new match trees.
This updates the filters panel to show only the org/name from the repo, removing the code host from the text by default and replacing it with a representative icon if it is a known codehost. Additionally, it adds a hover tooltip that shows the full repo name (including code host) so that a user can still see what repo it is even if it's a long repo name.
* don't allow redirect url when the request is from a mobile device
* add comment
* add comment
* BAZELLLLL
* disable authorize button on mobile devices
* add link to security issue
* update regex
* chore: drop unused go pkg
* go mod tidy + bazel run :gazelle-update-repos
---------
Co-authored-by: William Bezuidenhout <william.bezuidenhout@sourcegraph.com>
We received the feedback that it is not always obvious that metadata is line-separated in the CSV export of search results. Depending on the tool used to display the CSV, only the first metadata value might be visible. Additionally the current format can be hard to parse.
Here we add a new column "Repository metadata JSON" to the CSV export of search results, which encodes metadata into a single line JSON. We deprecate "Repository metadata" but keep it around for backward compatibility for short period of time.
## Test plan
- updated unit test
- manual testing:
- validated the JSON encoding with a python script that reads the CSV and parses the JSON field.
This does a few things to further clean up RepositoryResolver:
- It splits NewMinimalRepositoryResolver out of NewRepositoryResolver to make it more obvious when we're creating a fully-resolved RepositoryResolver and a lazy RepositoryResolver. It's pretty error prone to pass in a *types.Repo with only the Name and ID fields set.
- It moves the always-available fields ID and Name directly onto the struct to make it more clear that those are safe to use directly
- It refactors the lazy hydration to the more standard sync.Once pattern
- It changes a bunch of call sites to use api.RepoName instead of *types.Repo when a fully-resolved repo is not actually needed (removing a few DB calls in the process)
For reasons I don't fully understand even though it's probably my doing, RepositoryResolver depends on the search result type RepoMatch. There was some really convoluted logic about how only the name and id fields of RepoMatch were set, which is particularly confusing because a RepoMatch isn't even required to construct the RepositoryResolver. The only information being provided by RepoMatch was rev, which it really shouldn't be since a repository domain object should not even have a rev (that should be relegated to the GitCommitResolver or GitRefResolver, which contain a RepositoryResolver).
Anyways, this removes the dependency on the search result type.
Due to a newly enabled (in bazel 7) `--incompatible_sandbox_hermetic_tmp`, and nix setting TMP/TMPDIR to something below /tmp, we encounter https://github.com/bazelbuild/bazel/issues/5900
## Test plan
`nix develop --command bazel build //dev/sg` is successfull
This reverts the changes made to support glob filters in the new keyword
search patternType. We have had mixed feedback on the change, so would rather
focus on the core improvement which is interpretting via AND.
This commit was generated by reverting all commits specific to globbing:
- 14e382e568 search: respect patternType when navigating to search results (#59164)
- 4f6cdb658c search: don't convert rev to regex for kw search (#59165)
- c6f0aa4b5d search: mention glob syntax in search mode picker (#59168)
- a0e660e1a8 search: use literal suggestions for keyword search (#59120)
- 92cfe03c10 search: support glob syntax for repo and file filters (#59080)
Test Plan: CI and spun up in local dev. Browsed around with new keyword search
mode turned on checking we still used regex. Then did a few searches using
regex syntax and confirmed it worked.
