Part of
[GRAPH-759](https://linear.app/sourcegraph/issue/GRAPH-759/issue-with-apex-extension-not-appearing-for-langapex)
Linguist only supports a subset of the file extensions often used for
the Apex programming languages. This PR adds support for the main set
commonly used.
**Key changes**
1. Adds all extensions for Apex
2. Update our logic to handle multiple extensions for one language
3. Update tests to ensure we only manually map languages if they don't
exist OR have different extensions in go-enry (prevents us from
duplicating entries completely from go-enry)
## Test plan
- [x] Update unit tests
- [x] Validate locally by testing the language filter
If you look at the code for `bulkMonikerResultsQuery` and
`minimalBulkMonikerResultsQuery`, you'll see that the last
returned value is the `document_path`. The value returned
was directly used an `UploadRootRelPath` elsewhere. So it
makes sense to rename the field from URI to `DocumentPath`.
## Test plan
Covered by existing tests
The use of different types makes it clear which kind of path is needed
in which place. This also makes the CodeNavService layering clearer;
it has the responsibility of taking in RepoRelPaths and correctly interfacing
with LsifStore, which deals in UploadRelPath values.
- Updates enry to include languages from Linguist v7.29.0 such as Mojo etc.
- Updates auto-complete filters in frontend code.
- Updates Zoekt to pick up newer version with bumped enry dep.
- Updates language extension overrides to avoid ambiguity for `.json` and `.yml`.
- Updates snapshot tests.
This patch does a few things:
- Adds `go-enry` packages to depguard, so that people do not
accidentally use enry APIs instead of the corresponding APIs
in the `languages` package.
- Adds more tests for different functions in the languages package
to ensure mutual consistency in how language<->extension mappings
are handled.
- Adds tests for enry upgrades
- Adds comments with IDs so that related parts in the code can be
pieced together easily
Adds support to the lang file filter for Magik language (`lang: Magik`).
In order to do that we add wrappers around go-enry and update search
code to use them. This provides flexibility for us in the future to
support other languages that are not in Linguist as well.
'Dump' really meant an uploads that is a 'completed' state,
and hence is eligible for various operations such as 'Find references'
This patch makes that explicit by removing the term 'Dump'
from the backend in favor of 'CompletedUpload'
To determine the icon to show for a file, we need to know
the language. The code previously relied on the getFileInfo
API which did this using file extensions, leading to
incorrect results for header files with the '.h' extension,
which could have either C, C++ or Objective-C code.
Instead, we now get the language from the server,
where the go-enry library can optionally use the file
contents to return a more appropriate result.
Consequently, we can remove the getFileInfo API altogether.
Updates both the React and Svelte versions of the code.
Other changes in this patch:
- I've added icons for a bunch of languages which were missing.
- I've centralized the icon lists (and the bulk of the related logic) for
both React and Svelte in one place.
Now that we're running go 1.21, we can take advantage of its new features. This cleans up a few utilites that have since been included in the standard library.
* log: remove use of description paramter in Scoped
* temporarily point to sglog branch
* bazel configure + gazelle
* remove additional use of description param
* use latest versions of zoekt,log,mountinfo
* go.mod
Previously, we had a bunch of channels and goroutines, which
aren't necessary, since the invoking goroutine anyways processes
the documents one-by-one and writes them into the database
## Test plan
Covered by existing tests
Replaces an explicit `go:generate` call to `stringer` by its Bazel
counterpart.
This is a part of the effort to remove all `go:generate` statements and
to have them handled by Bazel. See tracked issue for more details.
## Test plan
<!-- All pull requests REQUIRE a test plan:
https://docs.sourcegraph.com/dev/background-information/testing_principles
-->
CI
Creates a `go_stringer` macro and replace lsif code that used a
`go:generate` + stringer. Because this generates Go code, it's still
better to have the generated code versioned, as it doesn't require
everyone to configure `gopls` so it plays nicely with Bazel.
This also introduces a `//dev:write_all` target that calls all known
targets that write back to the source tree.
## Test plan
<!-- All pull requests REQUIRE a test plan:
https://docs.sourcegraph.com/dev/background-information/testing_principles
-->
CI
The previous approach to enable race detection was too radical and
accidently led to build our binaries with the race flage enabled, which
caused issues when building images down the line.
This happened because putting a `test --something` in bazelrc also sets
it on `build` which is absolutely not what we wanted. Usually folks get
this one working by having a `--stamp` config setting that fixes this
when releasing binaries, which we don't at this stage, as we're still
learning Bazel.
Luckily, this was caught swiftly. The current approach insteads takes a
more granular approach, which makes the `go_test` rule uses our own
variant, which injects the `race = "on"` attribute, but only on
`go_test`.
## Test plan
<!-- All pull requests REQUIRE a test plan:
https://docs.sourcegraph.com/dev/background-information/testing_principles
-->
CI, being a main-dry-run, this will cover the container building jobs,
which were the ones failing.
---------
Co-authored-by: Alex Ostrikov <alex.ostrikov@sourcegraph.com>
- Write Rust one-for-one protocol compatible replacement for
universal-ctags in JSON streaming mode (scip-ctags)
- Use tree-sitter to generate scip symbols, then emit those through
scip-ctags
- These symbols will be reused for Cody context
- Ensure code is built with musl libc
## Test plan
Unit and snapshot tests in the Rust symbol generation code - verified
working in the symbols sidebar and symbol search for enabled languages.
---------
Co-authored-by: TJ DeVries <devries.timothyj@gmail.com>
Co-authored-by: William Bezuidenhout <william.bezuidenhout@sourcegraph.com>
Co-authored-by: Eric Fritz <eric@eric-fritz.com>
Co-authored-by: Eric Fritz <eric@sourcegraph.com>
Co-authored-by: Jean-Hadrien Chabran <jh@chabran.fr>
- Write Rust one-for-one protocol compatible replacement for
universal-ctags in JSON streaming mode (scip-ctags)
- Use tree-sitter to generate scip symbols, then emit those through
scip-ctags
- These symbols will be reused for Cody context
Currently, only zig is enabled (so other languages should remain unaffected by this change).
We will add other languages throughout the next week as we're able to check them off.
## Test plan
Unit and snapshot tests in the Rust symbol generation code - verified
working in the symbols sidebar and symbol search for enabled languages.
---------
Co-authored-by: SuperAuguste <19855629+SuperAuguste@users.noreply.github.com>
Co-authored-by: William Bezuidenhout <william.bezuidenhout@sourcegraph.com>
Co-authored-by: Eric Fritz <eric@eric-fritz.com>
Co-authored-by: Eric Fritz <eric@sourcegraph.com>
Due to an issue with Autogold and Bazel, we have to migrate all
`autogold` tests to use the `v2`. This draft PR loses some annotations
describing the test cases, but I'll update that after once I have
validate that we can get those to work with Bazel.
Why is this a huge PR and not a couple of smaller ones: Because this PR
needed a few manual fixes, it was simpler to grind through the changes
after running comby locally. I have attached the config below so you can
review what I used instead.
@chwarwick the code insights tests were heavily relying on the attached
description on `autogold.Want` to provide context, I manually ported
them by adding a new `name string` field on the test cases.
Comby config I used:
```
[update-imports]
match="\"github.com/hexops/autogold\""
rewrite="\"github.com/hexops/autogold/v2\""
[update-api-want]
match="autogold.Want(:[desc], :[v])"
rewrite="autogold.Expect(:[v])"
[update-api-equal]
match="autogold.Equal(:[v])"
rewrite="autogold.ExpectFile(:[v])"
```
```
$ comby -config ../autogold-comby.toml -matcher .go -exclude-dir vendor,node_modules -in-place
```
I then followed up with some manual fixes for the test cases that were
making the assumption that `want.Name()` exists.
## Test plan
<!-- All pull requests REQUIRE a test plan:
https://docs.sourcegraph.com/dev/background-information/testing_principles
-->
Green go tests.
---------
Co-authored-by: davejrt <davetry@gmail.com>
* Fix "Type can be omitted" warnings
* Fix "Redundant import alias" warnings
* Fix "Redundant type conversion" warnings
* Fix "Redundant parentheses" warnings
* Fix typo and case
Co-authored-by: Eric Fritz <eric@sourcegraph.com>
