Using Perforce depots with Sourcegraph

Sourcegraph supports Perforce depots using the git p4 adapter. This creates an equivalent Git repository from a Perforce depot. For Sourcegraph <3.25.1, src serve-git, Sourcegraph’s tool for serving local directories, is required. For Sourcegraph 3.25.1+ an experimental feature can be enabled to configure Perforce depots through the Sourcegraph UI.

Screenshot of using Sourcegraph for code navigation in a Perforce depot:

Viewing a Perforce repository on Sourcegraph

Sourcegraph 3.25.1+ configuration instructions

Adding Perforce depots through the UI is an experimental feature in Sourcegraph 3.25.1. To access this functionality, a site admin must enable the experimental feature in the site configuration:

{
	"experimentalFeatures": {
		"perforce": "enabled"
  }
  ...
}

To connect Perforce to Sourcegraph:

  1. Depending on whether you are a site admin or user:
    1. Site admin: Go to Site admin > Manage repositories > Add repositories
    2. User: Go to Settings > Manage repositories.
  2. Select Perforce.
  3. Configure the connection to Perforce using the action buttons above the text field, and additional fields can be added using Cmd/Ctrl+Space for auto-completion. See the configuration documentation below.
  4. Press Add repositories.

NOTE That adding code hosts as a user is currently in private beta.

Depot syncing

Use the depots field to configure which depots are mirrored/synchronized as Git repositories to Sourcegraph:

  • depots
    A list of depot paths that can be either a depot root or an arbitrary subdirectory.
  • p4.user
    The user to be authenticated for p4 CLI, and should be capable of performing p4 ping, p4 login, p4 trust and any p4 commands involved with git p4 clone and git p4 sync for listed depots. If repository permissions are mirrored, the user needs additional ability to perform the p4 protects, p4 groups, p4 group, p4 users commands (aka. “super” access level).
  • p4.passwd
    The ticket value to be used for authenticating the p4.user. It is recommended to create tickets of users in a group that never expire. Use the command p4 -u <p4.user> login -p -a to obtain a ticket value.

Notable things about depot syncing:

  • It takes approximately one second to import one Perforce change into a Git commit, this translates to sync a Perforce depot with 1000 changes takes approximately 1000 seconds, which is about 17 minutes. It is possible to limit the maximum changes to import using maxChanges config option.
  • Rename of a Perforce depot will cause a re-import of the depot, including changing the depot on the Perforce server or the repositoryPathPattern config option.

Repository permissions

To enable permissions syncing for Perforce depots by including the authorization field:

{
  ...
  "authorization": {}
}

Sourcegraph only supports repository-level permissions and does not match the granularity of Perforce access control lists (which supports file-level permissions). The workaround is for site admins to sync arbitrary subdirectories of a depot, which can then enforce permissions in Sourcegraph. We suggest using the most concrete path of your permissions boundary.

For example, if your Perforce depot //Sourcegraph/ has different permissions for //Sourcegraph/Backend/ and some subdirectories of //Sourcegraph/Frontend/, we recommend setting the following depots:

{
  ...
  "depots": [
    "//Sourcegraph/Backend/",
    "//Sourcegraph/Frontend/Web/",
    "//Sourcegraph/Frontend/Extension/"
  ]
}

By configuring each subdirectory that has unique permissions, Sourcegraph is able to recognize and enforce permissions for each defined repository.

Known limitations

Sourcegraph uses prefix-matching to determine if a user has access to a repository in Sourcegraph. That means if a user has access to a directory and also has exclusions to some subdirectories, those exclusions will not be enforced in Sourcegraph because Sourcegraph does not support file-level permissions.

For example, consider the following output of p4 protects -u alice:

list user * * -//...
list user * * -//spec/...
write user alice * //TestDepot/...
=write user alice * -//TestDepot/Secret/...

If the site admin configures "depots": ["//TestDepot/"], the exclusion of the last line will not be enforced in Sourcegraph. In other words, the user alice will have access to //TestDepot/Secret/ in Sourcegraph even though alice does not have access to this directory on the Perforce Server. To mitigate, use the most concrete path of your permissions boundary as described in the above section.

Besides, the host (fourth) value is disregarded from the output of p4 protects.

Configuration

admin/external_service/perforce.schema.json

{
	// If non-null, enforces Perforce depot permissions.
	"authorization": null,

	// Depots can have arbitrary paths, e.g. a path to depot root or a subdirectory.
	"depots": null,
	// Other example values:
	// - [
	//     "//Sourcegraph/",
	//     "//Engineering/Cloud/"
	//   ]

	// Only import at most n changes when possible (git p4 clone --max-changes).
	"maxChanges": 1000,

	// The ticket value for the user (P4PASSWD).
	"p4.passwd": null,

	// The Perforce Server address to be used for p4 CLI (P4PORT).
	"p4.port": null,
	// Other example values:
	// - "ssl:111.222.333.444:1666"

	// The user to be authenticated for p4 CLI (P4USER).
	"p4.user": null,
	// Other example values:
	// - "admin"

	// Rate limit applied when making background API requests to Perforce.
	"rateLimit": {
		"enabled": true,
		"requestsPerHour": 5000
	},

	// The pattern used to generate the corresponding Sourcegraph repository name for a Perforce depot. In the pattern, the variable "{depot}" is replaced with the Perforce depot's path.
	//
	// For example, if your Perforce depot path is "//Sourcegraph/" and your Sourcegraph URL is https://src.example.com, then a repositoryPathPattern of "perforce/{depot}" would mean that the Perforce depot is available on Sourcegraph at https://src.example.com/perforce/Sourcegraph.
	//
	// It is important that the Sourcegraph repository name generated with this pattern be unique to this Perforce Server. If different Perforce Servers generate repository names that collide, Sourcegraph's behavior is undefined.
	"repositoryPathPattern": "{depot}"
}

Sourcegraph <3.25.1 configuration instructions with src serve-git

Prerequisites

Create an equivalent Git repository and serve it to Sourcegraph

For each Perforce repository you want to use with Sourcegraph, follow these steps:

  1. Create a local Git repository with the contents of your Perforce depot: git p4 clone //DEPOT/[email protected] (replace //DEPOT/PATH with the Perforce repository path).
  2. Run src serve-git from the parent directory that holds all of the new local Git repositories.
  3. Follow the instructions in the src serve-git Quickstart to add the repositories to your Sourcegraph instance.

Updating Perforce depots

To update the repository after new Perforce commits are made, run git p4 sync in the local repository directory. These changes will be automatically reflected in Sourcegraph as long as src serve-git is running.

We recommend running this command on a periodic basis using a cron job, or some other scheduler. The frequency will dictate how fresh the code is in Sourcegraph, and can range from once every 10s to once per day, depending on how large your codebase is and how long it takes git p4 sync to complete.

Alternative to src serve-git: push the new Git repository to a code host

If you prefer, you can skip using src serve-git, and instead push the new local Git repository to a Git-based code host of your choice. For updates, you would run git p4 sync && git push periodically.

If you do this, the repositories you created on your Git host are normal Git repositories, so you can add the repositories to Sourcegraph as you would any other Git repositories.

Alternative for extra-large codebases

The instructions below will help you get Perforce depots on Sourcegraph quickly and easily, while retaining all code change history. If your Perforce codebase is large enough that converting it to Git takes long enough to cause noticeable staleness on Sourcegraph, you can use src-expose’s optional syncing functionality along with a faster fetching command (like p4 sync instead of git p4 sync) to periodically fetch and squash changes without trying to preserve the original Perforce history.

Known issues

We intend to improve Sourcegraph’s Perforce support in the future. Please file an issue to help us prioritize any specific improvements you’d like to see.

  • Sourcegraph was initially built for Git repositories only, so it exposes Git concepts that are meaningless for converted Perforce depots, such as the commit SHA, branches, and tags.
  • The commit messages for a Perforce depot converted to a Git repository have an extra line at the end with Perforce information, such as [git-p4: depot-paths = "//guest/acme_org/myproject/": change = 12345].