Bazel at Sourcegraph

Sourcegraph uses Bazel as its build system. Reach out on #ask-dev-experience for questions and support.

Overview

📅 Timeline

  • 2023-04-03 Bazel steps are required to pass on all PR builds.
    • Run bazel run //:gazelle-update-repos if you changed the go.mod.
    • Run bazel configure after making all your changes in the PR and commit the updated/added BUILD.bazel
    • Add [no-bazel] in your commit description if you want to bypass Bazel.
  • 2023-04-06 Bazel steps are required to pass on main.
    • Run bazel run //:gazelle-update-repos if you changed the go.mod.
    • Run bazel configure after making all your changes in the PR and commit the updated/added BUILD.bazel
    • Add [no-bazel] in your commit description if you want to bypass Bazel.
  • 2023-04-10 You cannot opt out of Bazel anymore
  • 2023-05-02 Additonally, to normal container building process, container images are now also built with rules_oci and Wolfi for as base images.
    • Those new images are used by the new test targets under //testing/... that run the E2E tests and Backend Integration tests.
    • The new images are deployed on S2 (and only S2).
    • See Slack thread
  • 2023-05-07 The new images are deployed on Dotcom.
  • 2023-05-14 After the release cut and the beginning of the code freeze, all operations that are now migrated to Bazel will have their legacy counterpart removed.

Bazel for teammates in a hurry

Bazel vocabulary

  • A rule is a function that stitches together parts of the graph.
    • ex: build go code
  • A target is a named rule invocation.
    • ex: build the go code for ./app
    • ex: run the unit tests for ./app
  • A package is a a group of targets.
    • ex: we only have one single package in the example above, the root one.

Bazel uses two types of files to define those:

  • WORKSPACE, which sits at the root of a project and tells Bazel where to find the rules.
    • ex: get the Go rules from that repository on GitHub, in this exact version.
  • BUILD.bazel, which sits in every folder that contains targets.

To reference them, the convention being used is the following: //pkg1/pkg2:my_target and you can say things such as //pkg1/... to reference all possible targets in a package.

Finally, let's say we have defined in our Bazel project some third party dependencies (a NPM module or a Go package), they will be referenced using the @ sign.

  • @com_github_keegancsmith_sqlf//:sqlf

Bazel cheat sheet

Keep in mind

  • Do not commit file whose name include spaces, Bazel does not like it.
  • Do not expect your tests to be executed inside the source tree and to be inside a git repository.
    • They will be executed in the sandbox. Instead create a temp folder and init a git repo manually over there.

Building and testing things

  • bazel build [path-to-target] builds a target.
    • ex bazel build //lib/... will build everything under the /lib/... folder in the Sourcegraph repository.
  • bazel test [path-to-target] tests a target.
    • ex bazel test //lib/... will run all tests under the /lib/... folder in the Sourcegraph repository.
  • bazel configure automatically inspect the source tree and update the buildfiles if needed.
  • bazel run //:gazelle-update-repos automatically inspect the go.mod and update the third parties dependencies if needed.

Debugging buildfiles

  • bazel query "//[pkg]/..." See all subpackages of pkg.
  • bazel query "//[pkg]:*" See all targets of pkg.
  • bazel query //[pkg] --output location prints where the buildfile for pkg is.
    • ex: bazel query @com_github_cloudflare_circl//dh/x448 --output location which allows to inspect the autogenerated buildfile.
  • bazel query "allpaths(pkg1, pkg2)" list all knowns connections from pkg1 to pkg2
    • ex bazel query "allpaths(//cmd/worker, @go_googleapis//google/api)"
    • This is very useful when you want to understand what connects a given package to another.

Running bazel built services locally with sg start

For early adopters only.

First you need to have bazel installed obviously, but also iBazel which will watch your files and rebuild if needed. We use a tool called bazelisk (which is also part of Bazel) to manage the version of bazel. It inspects a bunch of files to determine what bazel version to use for your repo.

If you want the setup automated run sg setup, otherwise you can install it manually by executing the following commands:

  • brew install bazelisk
  • brew install ibazel

Then instead of running sg start oss you can use the bazel variant instead.

  • sg start oss-bazel
  • sg start enterprise-bazel
  • sg start codeintel-bazel
  • sg start enterprise-codeintel-bazel
How it works

When sg start is booting up, the standard installation process will begin as usual for commands that are not built with Bazel, but you'll also see a new program running [ bazel] which will log the build process for all the targets required by your chosen commandset. Once it's done, these services will start and iBazel will take the relay. It will watch the files that Bazel has indentified as having dependencies for your services, and rebuild them accordingly.

So when a change is detected, iBazel will build the affected target and it will be restarted once the build finishes.

Caveats
  • You still need to run bazel configure if you add/remove files or packages.
  • Error handling is not perfect, so if a build fails, that might stop the whole thing. We'll improve this in the upcoming days, as we gather feedback.

Resources