Sourcegraph Instance Validation

Instance validation provides a quick way to check that a Sourcegraph instance functions properly after a fresh install or an update.

The src CLI has an experimental command validate which drives the validation from a user-provided configuration file with a validation specification (in JSON or YAML format). if no validation specification file is provided it will execute the following:

• temporarily add an external service
• wait for a repository to be cloned
• perform a search on the cloned repo
• perform a search on a non-indexed branch of the cloned repo
• remove the added external service

Validation specification

Validation specifications can be provided in either a YAML or JSON format. The best way to describe this initial, simple, and experimental validation specification is with the example below:

YAML File Specification

# creates the first admin user on a fresh install (skips creation if user exists)
firstAdmin:
    email: [email protected]
    username: foo
    password: "{{ .admin_password }}"

# adds the specified code host
externalService:
  config:
    url: https://github.com
    token: "{{ .github_token }}"
    orgs: []
    repos:
      - sourcegraph-testing/zap
  kind: GITHUB
  displayName: footest
  # set to true if this code host config should be deleted at the end of validation
  deleteWhenDone: true

# checks maxTries if specified repo is cloned and waits sleepBetweenTriesSeconds between checks 
waitRepoCloned:
  repo: github.com/sourcegraph-testing/zap
  maxTries: 5
  sleepBetweenTriesSeconds: 2

# performs the specified search and checks that at least one result is returned
searchQuery: 
  - repo:^github.com/sourcegraph-testing/zap$ test
  - repo:^github.com/sourcegraph-testing/[email protected] test

JSON File Specification

{   
   "firstAdmin": {
    "email": "[email protected]",
    "username": "foo",
    "password": "{{ .admin_password }}"
    },
    "externalService": {
        "config": {
            "url": "https://github.com",
            "token": "{{ .github_token }}",
            "orgs": [],
            "repos": [
                "sourcegraph-testing/zap"
            ]
        },
        "kind": "GITHUB",
        "displayName": "footest",
        "deleteWhenDone": true
    },
    "waitRepoCloned": {
        "repo": "github.com/sourcegraph-testing/zap",
        "maxTries": 5,
        "sleepBetweenTriesSeconds": 5
    },
    "searchQuery": [
        "repo:^github.com/sourcegraph-testing/zap$ test",
        "repo:^github.com/sourcegraph-testing/[email protected] test"
    ]
}

With this configuration, the validation command executes the following steps:

• create the first admin user
• add an external service
• wait for a repository to be cloned
• perform a search

Passing in secrets

It is often the case that the config file with the validation specification needs to declare passwords, tokens, or other secrets and these secrets should not be exposed or committed to a git repo.

The validation specification can refer to string values that come from a context specified outside the config file (see the Usage section below). References to string values from this outside context are specified like so: {{ .some_key }}. The context will have a string value defined under the key some_key and the validation execution will use that.

Usage

Use the src CLI to validate with a validation specification file:

src validate -context github_token=$GITHUB_TOKEN validate.yaml

src validate -context github_token=$GITHUB_TOKEN validate.json

To execute default validation checks:

src validate -context github_token=$GITHUB_TOKEN

The src binary finds the Sourcegraph instance to validate from the environment variables SRC_ENDPOINT and SRC_ACCESS_TOKEN.

Note: The SRC_ACCESS_TOKEN is not needed when a first admin user is declared in the validation specification.