Using Perforce depots with Sourcegraph

Sourcegraph supports Perforce Helix depots using p4-fusion. This creates an equivalent Git repository from a Perforce depot, which can then be indexed by Sourcegraph.

Screenshot of a Perforce repository in a Sourcegraph

Add a Perforce code host connection

Perforce depots can be added to a Sourcegraph instance by adding the appropriate code host connection.

Perforce code host connections are still an experimental feature.To enable Perforce code host connections, a site admin must:

  1. Enable the experimental feature in the site configuration:

    {
  "experimentalFeatures": {
    "perforce": "enabled"
  }
  // ...
}

    It is highly recommended that you also enable file-level permissions (step 1) at this time.
    Save your changes before continuing.

  2. Go to Site admin > Manage code hosts > Add code host

    you may need to manually refresh the page in order to display the Perforce option.

  3. Scroll down the list of supported code hosts and select Perforce.

  4. Configure which depots are mirrored/synchronized as Git repositories to Sourcegraph:

    • depots

      A list of depot paths that can be either a depot root or an arbitrary subdirectory. Note: Only "local" type depots are supported.

    • p4.user

      The user to be authenticated for p4 CLI, and should be capable of performing:

      • p4 login
      • p4 trust
      • and any p4 commands involved with git p4 clone and git p4 sync for listed depots.

      If repository permissions are mirrored, the user needs additional ability (aka. "super" access level) to perform the commands:

      • p4 protects
      • p4 groups
      • p4 group
      • p4 users

    • p4.passwd

      The ticket value to be used for authenticating the p4.user. It is recommended to create tickets of users in a group that never expire. Use the command p4 -u <p4.user> login -p -a to obtain a ticket value.

    • See the configuration documentation below for other fields you can configure.

  5. Configure fusionClient:

    {
  // ...
  "fusionClient": {
    "enabled": true,
    "lookAhead": 2000
  }
}

    Details of all p4-fusion configuration fields can be seen here.

  6. Click Add repositories.

Sourcegraph will now talk to the Perforce host and sync the configured depots to the Sourcegraph instance.

It's worthwhile to note some limitations of this process:

  • When syncing depots either git p4 or p4-fusion (recommended) are used to convert Perforce depots into git repositories so that Sourcegraph can index them.
  • Rename of a Perforce depot, including changing the depot on the Perforce server or the repositoryPathPattern config option, will cause a re-import of the depot.
  • Unless permissions syncing is enabled, Sourcegraph is not aware of the depot permissions, so it can't enforce access restrictions.

Repository permissions

To enforce repository-level permissions for Perforce depots using Perforce permissions tables, include the authorization field in the configuration of the Perforce code host connection you created above:

{
  // ...
  "authorization": {}
}

Adding the authorization field to the code host connection configuration will enable partial parsing of the permissions tables. Learn more about the partial support of permission table parsing.

Syncing subdirectories to match permission boundaries

By default Sourcegraph only supports repository-level permissions and does not match the granularity of Perforce permissions tables.

If you don't activate file-level permissions you should sync subdirectories of a depot using the depots configuration that best describes the most concrete path of your permissions boundary.

For example, if your Perforce depot //depot/Talkhouse has different permissions for //depot/Talkhouse/main-dev and subdirectories //depot/Talkhouse/rel1.0/front, //depot/Talkhouse/rel1.0/back we recommend setting the following depots:

{
  // ...
  "depots": [
    "//depot/Talkhouse/main-dev",
    "//depot/Talkhouse/rel1.0/front",
    "//depot/Talkhouse/rel1.0/back"
  ]
}

By configuring each subdirectory that has unique permissions, Sourcegraph is able to recognize and enforce permissions for the sub-directories. You can NOT define these permissions as:

{
  // ...
  "depots": [
    "//depot/Talkhouse/main-dev",
    "//depot/Talkhouse/rel1.0",
    "//depot/Talkhouse/rel1.0/back"
  ]
}

Since that would override the permissions for the //depot/Talkhouse/rel1.0/back depot.

Wildcards

File-level permissions can handle wildcards in the protections table.If file-level permissions is not enabled, Sourcegraph provides limited support for * and ... paths, so the workaround of adding sub-folders as separate repositories for the paths that employ wildcards needs to be followed.

File-level permissions

File-level permissions make the syncing of subdirectories to match permission boundaries unnecessary.

To enable file-level permissions:

  1. Enable the feature in the site config:

    {
  // ...
  "experimentalFeatures": {
    "perforce": "enabled",
    "subRepoPermissions": { "enabled": true }
  }
}

  2. Enable the feature in the code host configuration by adding subRepoPermissions to the authorization object:

    {
  // ...
  "authorization": {
    "subRepoPermissions": true
  }
}

  3. Save the configuration. Permissions will be synced in the background based on your Perforce permissions tables.

Notes about permissions

  • Sourcegraph users are mapped to Perforce users based on their verified email addresses.
  • As long as a user has been granted at least Read permissions in Perforce they will be able to view content in Sourcegraph.
  • As a special case, commits in which a user does not have permissions to read any files are hidden. If a user can read a subset of files in a commit, only those files are shown.
  • The host field from protections are not supported.

Configuration

admin/external_service/perforce.schema.json 

{
	// If non-null, enforces Perforce depot permissions.
	"authorization": null,

	// Depots can have arbitrary paths, e.g. a path to depot root or a subdirectory.
	"depots": null,
	// Other example values:
	// - [
	//     "//Sourcegraph/",
	//     "//Engineering/Cloud/"
	//   ]

	// Configuration for the experimental p4-fusion client
	"fusionClient": null,

	// Only import at most n changes when possible (git p4 clone --max-changes).
	"maxChanges": 1000,

	// Client specified as an option for p4 CLI (P4CLIENT, also enables '--use-client-spec')
	"p4.client": null,

	// The ticket value for the user (P4PASSWD). You can get this by running `p4 login -p` or `p4 login -pa`. It should look like `6211C5E719EDE6925855039E8F5CC3D2`.
	"p4.passwd": null,

	// The Perforce Server address to be used for p4 CLI (P4PORT).
	"p4.port": null,
	// Other example values:
	// - "ssl:111.222.333.444:1666"

	// The user to be authenticated for p4 CLI (P4USER).
	"p4.user": null,
	// Other example values:
	// - "admin"

	// Rate limit applied when making background API requests to Perforce.
	"rateLimit": {
		"enabled": true,
		"requestsPerHour": 5000
	},

	// The pattern used to generate the corresponding Sourcegraph repository name for a Perforce depot. In the pattern, the variable "{depot}" is replaced with the Perforce depot's path.
	//
	// For example, if your Perforce depot path is "//Sourcegraph/" and your Sourcegraph URL is https://src.example.com, then a repositoryPathPattern of "perforce/{depot}" would mean that the Perforce depot is available on Sourcegraph at https://src.example.com/perforce/Sourcegraph.
	//
	// It is important that the Sourcegraph repository name generated with this pattern be unique to this Perforce Server. If different Perforce Servers generate repository names that collide, Sourcegraph's behavior is undefined.
	"repositoryPathPattern": "{depot}"
}

Known issues and limitations

We are actively working to significantly improve Sourcegraph's Perforce support. Please file an issue to help us prioritize any specific improvements you'd like to see.

  • Sourcegraph was initially built for Git repositories only, so it exposes Git concepts that are meaningless for converted Perforce depots, such as the commit SHA, branches, and tags.
  • The commit messages for a Perforce depot converted to a Git repository have an extra line at the end with Perforce information, such as [git-p4: depot-paths = "//guest/example_org/myproject/": change = 12345].
  • Permissions