How to do one-off profiling for dogfood and production using pprof

Go has built-in support for a sampling profiler pprof, which can be used to investigate CPU usage, heap usage and more.

You can obtain and examine a profile from a running Sourcegraph instance as follows:

  1. Select the right Kubernetes context using kubectx or kubectl.
  2. Set up port-forwarding for the pod you’re interested in profiling using k9s or kubectl. For example: kubectl port-forward precise-code-intel-worker-0000000000-00000 6061:6060 This will map port 6060 on the pod to port 6061 on your machine.
  3. Record a profile. # Sample CPU usage for 60 seconds go tool pprof -seconds 60 http://localhost:6061/debug/pprof/profile # Sample heap usage for 60 seconds go tool pprof -seconds 60 http://localhost:6061/debug/pprof/heap This will save the output to a temporary file. (Or you can specify a path using -output <path>.)
  4. Examine the generated using go tool pprof: # in the web UI go tool pprof -http :9999 /path/to/profile.pb.gz # in a REPL go tool pprof /path/to/profile.pb.gz The web UI supports visualizing the output as a flamegraph, as a call graph with weighted edges, and more.

Other resources: * (Blog post) Profiling Go tools with pprof * pprof documentation * net/http/pprof documentation * runtime/pprof documentation * How to enable continuous profiling in production