To run tests for the Go backend, run go test ./..., or specify a package
directly, go test ./util/textutil.
yarn in the Sourcegraph root directory if it is a fresh clone.yarn test from the root directory.yarn test --watch.
jest.autoEnable: true (and, if you want, jest.showCoverageOnLoad: true)test('name ...', ...) definition in your test file (be sure to set a breakpoint or break on uncaught exceptions by clicking in the left gutter).yarn test from any of the individual project dirs (shared/, web/, browser/).Usually while developing you will either have yarn test --watch running in a terminal or you will use vscode-jest.
React component snapshot tests are one way of testing React components. They make it easy to see when changes to a React component result in different output. Snapshots are files at __snapshots__/MyComponent.test.tsx.snap relative to the component’s file, and they are committed (so that you can see the changes in git diff or when reviewing a PR).
react-test-renderer for usage examples.yarn test --updateSnapshot) to update all snapshot files. Be sure to review the diff!E2e tests act like a user by opening a browser and clicking, typing, and navigating around in an automated fashion. They test the whole app: JS, CSS, and backend.
When an e2e test fails (example), CI displays a snapshot of the failure inline in the Buildkite output and Jest prints the test name, the error, and the line of code that failed:
A video of the session is available in the Artifacts tab:
Here are common failure modes:
sourcegraph/server container failed to start, so check the container logs that appear further down in the Buildkite outputwaitForSelector(selector, { visible: true })page.close() asynchronously, the browser crashed (check the video), or the build got canceledretry()retry is the preferred way to “poll” for a condition that cannot be expressed through waitForSelector() (as opposed to relying on a fixed setTimeout())Retrying the Buildkite step can help determine whether the test is flaky or broken. If it’s flaky, disable it with test.skip and file an issue on the author.
To run all e2e tests locally against your dev server, create a user test with password testtesttest, promote as site admin, then run:
env TEST_USER_PASSWORD=testtesttest GITHUB_TOKEN=<token> yarn --cwd web run test-e2e
There’s a GitHub test token in ../dev-private/enterprise/dev/external-services-config.json.
This will open Chromium, add a code host, clone repositories, and execute the e2e tests.
During a test run, the console from the browser will also be printed to the terminal, prefixed with “🖥 Browser console:”. Not every browser error log indicates a failure, but it can be helpful in debugging. Make sure to always first look at the test failure at the bottom of the logs, which includes the error message and stack trace.
To stop the test run on the first failing test, append --bail to your command.
You can find a complete list of all possible options in the Mocha documentation.
When a test fails, a screenshot is saved to the ./puppeteer directory.
In iTerm (macOS) and on Buildkite, it is also displayed inline in the terminal.
This may trigger a prompt “Allow Terminal-initiated download?” in iTerm.
Tick “Remember my choice” and click “Yes” if you want the inline screenshots to show up.
You can single-out one test with test.only:
test.only('widgetizes quuxinators', async () => { // ... })
Alternatively, you can use -g to filter tests: env ... test-e2e -g "some test name".
For regression tests, you can also run tests selectively with a command like yarn run test:regression:search, which runs the tests for search functionality.
See the test files for the environments and repositories that are cloned for tests.
If CI appears stuck on e2e tests, you can view the screen in VNC Viewer (free) by forwarding port 5900 to the pod. Find the pod name on the top right of the step in Buildkite:
You might have to inspect element to view it.
Drop the -N suffix from the name, then run:
gcloud container clusters get-credentials ci --zone us-central1-a --project sourcegraph-dev kubectl port-forward -n buildkite <buildkite agent pod> 5900:5900
Open VNC Viewer and type in localhost:5900. Hit Enter and accept the warning. Now you’ll be able to see what’s causing the tests to hang (e.g. a prompt or alert that hasn’t been dismissed).
Open web/src/e2e/e2e.test.ts and add a new test:
test('widgetizes quuxinators', async () => { await page.goto(baseURL + '/quuxinator/widgetize') await page.waitForSelector('.widgetize', { visible: true }) // ... })
The full Puppeteer API is quite large, but most tests only use a few common commands:
await page.goto(baseURL + '/some/route') navigate to a URLawait page.waitForSelector(selector, { visible: true }) wait for an element to appearawait page.click(selector) click on an element (must be visible, but not necessarily in the viewport)The easiest way to write CSS selectors is to inspect the element in your browser and look at the CSS classes. From there, you can write a selector and get immediate feedback:
CSS selectors in e2e tests should always refer to CSS classes prefixed with e2e-. This makes them easy to spot in the implementation and therefor less likely to accidentally break. e2e- classes are never referenced in stylesheets, they are added in addition to styling classes. If an element you are trying to select does not have an e2e- class yet, modify the implementation to add it.
If the element you are trying to select appears multiple times on the page (e.g. a button in a list) and you need to select a specific instance, you can use data-e2e-* attributes in the implementation:
<div data-e2e-item-name={this.props.name}> <span>{this.props.name}</span> <button className="e2e-item-delete-button">Delete</button> </div>
Then you can select the button with [data-e2e-item-name="foo"] .e2e-item-delete-button.
Important: it’s generally unreliable to hold references to items that are acted upon later. In other words, don’t do this:
const elem = page.selector(".selector") elem.click()
Do this:
page.click(".selector")
In the testing pyramid, e2e tests account for a small minority of all of the tests in an app. Only reach for e2e testing when it’s too difficult to unit test something.
In comparison to unit tests, e2e tests are slower and flakier but often more convenient.
E2e tests are typically beneficial for testing a happy-path through a user flow, such as adding a repository then running a search.
E2e tests are probably not worth the slowness/flakiness for testing a matrix of inputs, benchmarking, checking DOM structure, or verifying the correctness of logic.
When you submit a PR, a check from https://percy.io/Sourcegraph/Sourcegraph will appear:
If Percy failed CI ❌ then click on the Details link to review the visual changes:
Click the image on the right to toggle between diff and full image mode to review the change. Diff mode shows the changes in red.
If the changes are intended, click Approve 👍
Once you approve all of the changes, the Percy check will turn green ✅
Open web/src/e2e/index.e2e.test.tsx and add a new e2e test:
test('Repositories list', async () => { await page.goto(baseURL + '/site-admin/repositories?query=gorilla%2Fmux') await page.waitForSelector('[e2e-repository-name="/github.com/gorilla/mux"]', { visible: true }) await percySnapshot(page, 'Repositories list') })
The percySnapshot() function takes the snapshot and uploads it to Percy.io.
When you submit the PR, Percy will fail until you approve the new snapshot.
Flakiness in snapshot tests can be caused by the search response time, order of results, animations, premature snapshots while the page is still loading, etc.
This can be solved with Percy specific CSS that will be applied only when taking the snapshot and allow you to hide flaky elements with display: none.
The test suite is exercised on every pull request. For the moment CI output access is limited to Sourcegraph employees, though we hope to enable public read-only access soon.
The test pipeline is generated by dev/ci/gen-pipeline.go, and written to a
YAML file by dev/ci/init-pipeline.yml. This pipeline is immediately scheduled by
Buildkite to run on the Sourcegraph build farm. Some things that are tested
include:
When a pull request is coming from a non-Sourcegrapher, it won’t trigger a CI build on Buildkite automatically because we want to review the code before it runs on our CI infrastructure. Please review the PR to ensure it doesn’t make any malicious changes to our build scripts.
Here are the steps to manually trigger a build from the web (i.e. https://buildkite.com):
ae724a83f8b6fc5628a4e8efcbb62975ed7b4c33 for #8234.8160-http-warning (trim Akarshit: prefix) for #8234.To manually test against a Kubernetes cluster, use https://k8s.sgdev.org.
For testing with a single Docker image, run something like
IMAGE=sourcegraph/server:3.14.1 ./dev/run-server-image.sh