Automatic multi-version upgrades
From Sourcegraph 5.1 and later, multi-version upgrades can be performed automatically as if they were a standard upgrade for the same deployment type. Automatic multi-version upgrades take the following general form:
- Determine if your instance is ready to Upgrade:
- Merge the latest Sourcegraph release into your deployment manifests.
- With upstream changes to your manifests merged, start the new instance.
The upgrade magic now happens when the new version is booted. In more detail, starting a new
frontend container will:
- Detect that a previous version of Sourcegraph was/is currently running.
- Plan and persist database schema migrations based on the previously running version, the new target version, and the database state.
- Start a new internal server that sends poison pills to disconnect old services from the databases and prevents new services from connecting before the upgrade completes.
- Start a status server in place of the
frontendservice's usual primary exposed port
- Runs the migration plan (performing the same steps as
migration upgrade ...)
- Shuts down the internal and status servers and continues to boot normally
During an automatic multi-version upgrade, we'll attempt to boot a status server in the frontend container that is running (or blocking) on an active upgrade attempt. If there is an upgrade failure that affects the frontend, this status page will not be available and the
frontend container logs should be viewed. Optimistically, the status server will also be unreachable in the case that an upgrade performs quickly enough that there's no time for the status server to start.
In the case that there's a migration failure, or an unfinished out-of-band migration that needs to be complete, the status server will be served instead of the normal Sourcegraph React app. The following screenshots show an upgrade from Sourcegraph v3.37.1 to Sourcegraph 5.0, in which the
frontend schema is applying (or waiting to apply) a set of schema migrations, the
codeintel schema has a pair of schema migration failures, and a single unfinished out-of-band migration is still actively being performed to completion.
Pre v5.0.0 Automatic Multiversion Upgrades
Whether or not a Sourcegraph instance will perform an automatic upgrade is determined by the state of the
versions table in the
pgsql (also called
frontend) database. After
v5.0.0 a column exists on
auto_upgrade set either true of false, this controls whether or not automatic upgrades will be attempted by a Sourcegraph instance.
For versions prior to v5.0.0 the
SRC_AUTOUPGRADE environment variable may be set on the
migrator service manifests. To enable automatic upgrades. Ex:
sourcegraph-frontend-0: container_name: sourcegraph-frontend-0 image: 'index.docker.io/sourcegraph/frontend:4.5.1@sha256:22bb1203a6d8ac9bab442dcfef867efc216181026c3d6fc62415ef1a3f063139' cpus: 4 mem_limit: '8g' environment: - SRC_AUTOUPGRADE=true - DEPLOY_TYPE=docker-compose - 'OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4317' - PGHOST=pgsql - CODEINTEL_PGHOST=codeintel-db - CODEINSIGHTS_PGDATASOURCE=postgres://postgres:password@codeinsights-db:5432/postgres - 'SRC_GIT_SERVERS=gitserver-0:3178' - 'SRC_SYNTECT_SERVER=http://syntect-server:9238'
Drift and Automatic Upgrades
Multiversion upgrades should be performed only when no drift is detected in your database, and by default if database drifts are detected the migrator
upgrade command will not perform an upgrade. In some cases however drift may be expected, in such cases the automatic multiversion upgrade drift check may be bypassed by setting the
SRC_AUTOUPGRADE_IGNORE_DRIFT environment variable on