sourcegraph/enterprise/cmd/frontend/internal/executorqueue/queues/codeintel/queue.go

28 lines
1.2 KiB
Go
Raw Normal View History

package codeintel
import (
"context"
"github.com/sourcegraph/sourcegraph/enterprise/cmd/frontend/internal/executorqueue/handler"
"github.com/sourcegraph/sourcegraph/enterprise/internal/codeintel/autoindexing"
"github.com/sourcegraph/sourcegraph/enterprise/internal/codeintel/shared/types"
apiclient "github.com/sourcegraph/sourcegraph/enterprise/internal/executor"
"github.com/sourcegraph/sourcegraph/internal/database"
"github.com/sourcegraph/sourcegraph/internal/observation"
dbworkerstore "github.com/sourcegraph/sourcegraph/internal/workerutil/dbworker/store"
)
Fixup executor singlebinary refactor issues (#46721) This PR aims to address the feedback on the initial #app land on main. I've added a standalone version of the `executor run` command function body, so we don’t run an actual CLI in there and parse cmd line args as well, which would've meant running `sourcegraph test-vm` would actually attempt to spawn a test-vm and os.Exit after - probably not desired. Then I could restore the original main function we initially had before the shared package refactor, because executor is unlike all our other services, not only a service but a CLI tool, and only one of the subcommands is a service. This is already shared and can be invoked from singlebinary now. So that made the shared.Main superfluous, so I renamed the package to `singlebinary` and added a comment to never use that outside of it, to make the purpose clear. ..And finally added an in-memory secret to address the security concerns that I raised in slack before. It seems that the app stack doesn't yet fully work in local dev, so I cannot test it, but I think this is the right way to go about it and we can follow up with a fix if required once the general stack is working properly. ## Test plan Verified executor works in `sg run batches` again without error, that the `executor` cli tool properly exists after a command finished (which caused problems in CI too with it hanging forever), and tried to see if the app doesn't log worse errors than before, which doesn't seem to be the case. Also ran a main-dry-run, a regular build, and just to be extra sure an executor-patch-notest CI job as well. This seems to fix the red builds on main that we've seen this morning.
2023-01-20 17:10:46 +00:00
func QueueOptions(observationCtx *observation.Context, db database.DB, accessToken func() string) handler.QueueOptions[types.Index] {
recordTransformer := func(ctx context.Context, _ string, record types.Index, resourceMetadata handler.ResourceMetadata) (apiclient.Job, error) {
Fixup executor singlebinary refactor issues (#46721) This PR aims to address the feedback on the initial #app land on main. I've added a standalone version of the `executor run` command function body, so we don’t run an actual CLI in there and parse cmd line args as well, which would've meant running `sourcegraph test-vm` would actually attempt to spawn a test-vm and os.Exit after - probably not desired. Then I could restore the original main function we initially had before the shared package refactor, because executor is unlike all our other services, not only a service but a CLI tool, and only one of the subcommands is a service. This is already shared and can be invoked from singlebinary now. So that made the shared.Main superfluous, so I renamed the package to `singlebinary` and added a comment to never use that outside of it, to make the purpose clear. ..And finally added an in-memory secret to address the security concerns that I raised in slack before. It seems that the app stack doesn't yet fully work in local dev, so I cannot test it, but I think this is the right way to go about it and we can follow up with a fix if required once the general stack is working properly. ## Test plan Verified executor works in `sg run batches` again without error, that the `executor` cli tool properly exists after a command finished (which caused problems in CI too with it hanging forever), and tried to see if the app doesn't log worse errors than before, which doesn't seem to be the case. Also ran a main-dry-run, a regular build, and just to be extra sure an executor-patch-notest CI job as well. This seems to fix the red builds on main that we've seen this morning.
2023-01-20 17:10:46 +00:00
return transformRecord(ctx, db, record, resourceMetadata, accessToken())
}
store := dbworkerstore.New(observationCtx, db.Handle(), autoindexing.IndexWorkerStoreOptions)
return handler.QueueOptions[types.Index]{
Name: "codeintel",
Store: store,
RecordTransformer: recordTransformer,
}
}