gqltest: add docs to run and add tests (#12285)

Co-authored-by: Rijnard van Tonder <rvantonder@gmail.com>

.github/CODEOWNERS

@@ -133,9 +133,9 @@
 /internal/tracer/ @sourcegraph/cloud
 /internal/vcs/ @sourcegraph/cloud
 
-# Backend e2e tests
-/dev/e2e/ @unknwon
-/internal/e2eutil/ @unknwon
+# Backend integration tests
+/dev/gqltest/ @unknwon
+/internal/gqltestutil/ @unknwon
 
 # Campaigns
 /cmd/frontend/graphqlbackend/campaigns.go @sourcegraph/campaigns

dev/gqltest/README.md

@@ -0,0 +1,76 @@
+# GraphQL integration tests
+
+This directory contains GraphQL-based integration tests in the form of standard Go tests.
+
+## How to set up credentials
+
+Tests use environment variables to accept credentials of different external services involved, suggest to use [direnv](https://direnv.net/) to presistent those credentials for your convenience. Here is a comperhensive example `.envrc` file (you're free to use any other means, e.g. `.profile` or `.bashrc`):
+
+```sh
+# Your GitHub personal access token, this token needs to have scope to access private
+# repositories of "sgtest" organization. If you haven't joined "sgtest" organization,
+# please post a message on #dev-chat to ask for an invite.
+export GITHUB_TOKEN=<REDACTED>
+
+# Please go to https://team-sourcegraph.1password.com/vaults/dnrhbauihkhjs5ag6vszsme45a/allitems/5q5lnpirajegt7uifngeabrak4
+# and copy relevant credentials to here.
+export AWS_ACCESS_KEY_ID=<REDACTED>
+export AWS_SECRET_ACCESS_KEY=<REDACTED>
+export AWS_CODE_COMMIT_USERNAME=<REDACTED>
+export AWS_CODE_COMMIT_PASSWORD=<REDACTED>
+
+export BITBUCKET_SERVER_URL=<REDACTED>
+export BITBUCKET_SERVER_TOKEN=<REDACTED>
+export BITBUCKET_SERVER_USERNAME=<REDACTED>
+```
+
+You need to run `direnv allow` after editing the `.envrc` file.
+
+## How to run tests
+
+GraphQL-based integration tests are running against a live Sourcegraph instance, the eaiset way to make one is by booting up a single Docker container:
+
+```sh
+# For easier testing, run Sourcegraph instance without volume,
+# so it always starts from a clean state.
+docker run --publish 7080:7080 --rm sourcegraph/server:insiders
+```
+
+Once the the instance is live (look for the log line `✱ Sourcegraph is ready at: http://127.0.0.1:7080`), you can open another terminal tab to run these tests under this directory (`dev/gqltest`):
+
+```sh
+→ go test -tags "gqltest"
+2020/07/17 14:17:32 Site admin has been created: gqltest-admin
+PASS
+ok  	github.com/sourcegraph/sourcegraph/dev/gqltest	31.521s
+```
+
+### Testing against local dev instance
+
+It is not required to boot up a single Docker container to run these tests, which means it's also possible to run these tests against any Sourcegraph instance, for example, your local dev instance:
+
+```sh
+go test -tags "gqltest" -base-url "http://localhost:3080" -email "joe@sourcegraph.com" -username "joe" -password "<REDACTED>"
+```
+
+Generally, you're able to repeatedly run these tests regardless of any failures because tests are written in the way that cleans up and restores to the previous state. It is aware of if the instance has been initialized, so you can focus on debugging tests.
+
+Because we're using the standard Go test framework, you are able to just run a single or subset of these tests:
+
+```sh
+→ go test -tags "gqltest" -run TestSearch
+2020/07/17 14:20:59 Site admin authenticated: gqltest-admin
+PASS
+ok  	github.com/sourcegraph/sourcegraph/dev/gqltest	3.073s
+```
+
+## How to add new tests
+
+Adding new tests to this test suite is as easy as adding a Go test, here are some general rules to follow:
+
+- Add `// +build gqltest` on every new file as the very first line.
+- Use `gqltest-` prefix for entity name, and be as specific as possible for easier debugging, e.g. `gqltest-org-user-1`.
+- Restore the previous state regardless of failures, including:
+  - Delete new users created during the test.
+  - Delete external service created during the test.
+  - Although, sometimes you would not want to delete an entity so you could login and inspect the failure state.

dev/gqltest/main_test.go

@@ -15,11 +15,6 @@ import (
 	"github.com/sourcegraph/sourcegraph/internal/gqltestutil"
 )
 
-/*
-	NOTE: For easier testing, run Sourcegraph instance without volume:
-			docker run --publish 7080:7080 --rm sourcegraph/server:insiders
-*/
-
 var client *gqltestutil.Client
 
 var (