How to set up Telemetry Gateway locally

Telemetry Gateway is a managed service that ingests events exported from Sourcegraph instances, manipulates them as needed, and exports them to designated Pub/Sub topics or other destinations for processing.

It exposes a gRPC API defined in telemetrygateway/v1, and the service itself is implemented in cmd/telemetry-gateway.

To learn more about the Sourcegraph's new Telemetry framework, refer to the telemetry documentation.

Default development behaviour

A test deployment is available at telemetry-gateway.sgdev.org (see go/msp-ops/telemetry-gateway#dev), which publishes events to a test topic and development pipeline - currently sourcegraph-telligent-testing/event-telemetry-test. This instance only accepts licensed instance events that use a development-only license key, and is continuously deployed using MSP rollouts.

Exports of V2 telemetry events to this development instance is enabled by default in development using the TELEMETRY_GATEWAY_EXPORTER_EXPORT_ADDR environment variable configured in sg.config.yaml - for example, sg start will export V2 telemetry events to this instance.

Running Telemetry Gateway locally

First, start a Telemetry Gateway instance locally:

sg run telemetry-gateway

Then, configure the TELEMETRY_GATEWAY_EXPORTER_EXPORT_ADDR environment variable in sg.config.overwrite.yaml to send events to this locally running instance:

env:
  TELEMETRY_GATEWAY_EXPORTER_EXPORT_ADDR: 'http://127.0.0.1:6080'

By default, the local Telemetry Gateway instance will simply log any events it receives at debug level without forwarding the events anywhere. To see the message payloads it would emit in a production environment, configure the log level in sg.config.overwrite.yaml as well:

commands:
  telemetry-gateway:
    env:
      SRC_LOG_LEVEL: debug

You can increase the frequency of exports to monitor behaviour closer to real-time by setting the following in sg.config.yaml:

env:
  TELEMETRY_GATEWAY_EXPORTER_EXPORT_INTERVAL: "10s"

In development, a gRPC interface is enabled for Telemetry Gateway as well at http://127.0.0.1:10085/debug/grpcui/.