Kubernetes on Amazon EKS

Amazon EKS is Amazon's managed Kubernetes offering, similar to how Google Cloud offers managed Kubernetes clusters (GKE).

If your preferred cloud provider is Amazon, we strongly recommend using EKS instead of plain EC2. By using EKS, you will not need to manage your own Kubernetes control plane (complex). Instead, Amazon will provide it for you and you will only be responsible for managing Sourcegraph, which runs on the Kubernetes cluster.

Create the Amazon EKS Service Role

Follow the EKS Getting Started guide to create the IAM EKS service role:

Open the IAM console.
Click Roles -> Create role.
Choose EKS, accept the defaults and Next: Permissions.
Click Next: Review.
Under Role name, enter eksServiceRoleSourcegraph, then Create role.

Create the Amazon EKS Cluster VPC

Open the AWS CloudFormation console.
Ensure the region in the top right navigation bar is an EKS-supported region (many regions do not support EKS yet - see this list for supported regions).
Click Create stack, and select "with new resources".

When prompted to specify a template, select "Amazon S3 URL" as your Template Source and enter:

https://amazon-eks.s3-us-west-2.amazonaws.com/cloudformation/2020-04-21/amazon-eks-vpc-sample.yaml

Under Stack name, enter eks-vpc-sourcegraph.
Click Next through the following pages until you get the option to Create stack. Review the configuration and click Create stack.

For more details on these steps, refer to Amazon EKS prerequisites: Create your Amazon EKS cluster VPC.

Create the Amazon EKS Cluster

Open the EKS console.
Click Create cluster.
Under Cluster name, enter sourcegraph.
Under Cluster Service Role, select eksServiceRoleSourcegraph.
Under VPC, select eks-vpc-sourcegraph.
Under Security groups, select the one prefixed eks-vpc-sourcegraph-ControlPlaneSecurityGroup-. (Do NOT select NodeSecurityGroup.)
Accept all other values as default and click Create.
Wait for the cluster to finish CREATING. This will take around 10 minutes to complete, so grab some ☕.

For more details on these steps, refer to Amazon EKS prerequisites: Create your Amazon EKS cluster.

Create Kubernetes cluster worker nodes

Open the AWS CloudFormation console.
Click Create stack

Select the very last Specify an Amazon S3 template URL option and enter

https://amazon-eks.s3.us-west-2.amazonaws.com/cloudformation/2020-04-21/amazon-eks-nodegroup.yaml

Under Stack name, enter sourcegraph-worker-nodes.
Under ClusterName, enter the exact cluster name you used (sourcegraph).
Under ClusterControlPlaneSecurityGroup, scroll down or begin typing and select the option prefixed eks-vpc-sourcegraph-ControlPlaneSecurityGroup- (Do NOT select the NodeSecurityGroup.)
Under NodeGroupName, enter sourcegraph-node-group.
Choose NodeAutoScalingGroupMinSize and NodeAutoScalingGroupMaxSize and NodeInstanceType based on the following chart:

Users	Instance type	Min nodes	Max nodes	Cost est.	Attached Storage	Root Storage
10-500	m5.4xlarge	3	6	$59-118/day	500 GB	100 GB
500-2000	m5.4xlarge	6	10	$118-195/day	500 GB	100 GB

Note: You can always come back here later and modify these values to scale up/down the number of worker nodes. To do so, just visit the console page again, select Actions, Create Change Set For Current Stack, enter the same template URL mentioned above, modify the values and hit "next" until reviewing final changes, and finally Execute.

Under KeyName, choose a valid key name so that you can SSH into worker nodes if needed in the future.
Under VpcId, select eks-vpc-sourcegraph-VPC.
Under Subnets, search for and select all eks-vpc-sourcegraph subnets.
Click Next through the following pages until you get the option to Create stack. Review the configuration and click Create stack.

For more details on these steps, refer to Worker Nodes: Amazon EKS-optimized Linux AMI.

Install `kubectl` and configure access to the cluster

On your dev machine:

Install the aws CLI tool: bundled installer, other installation methods.
Follow these instructions to create an access key and aws configure the CLI to use it.
Install kubectl and aws-iam-authenticator by following these steps.

Configure kubectl to interact with your cluster:

aws eks update-kubeconfig --name ${cluster_name}

Important: If kubectl commands prompt you for username/password, be sure that kubectl version reports a client version of v1.10+. Older versions of kubectl do not work with the authentication configuration provided by Amazon EKS.

At this point, kubectl get svc should show something like:

$ kubectl get svc
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   172.20.0.1   <none>        443/TCP   4m

Enable worker nodes to join the Kubernetes cluster

Now it is time to enable the worker nodes created by CloudFormation to actually join the Kubernetes cluster:

Download, edit, and save this configuration map file:

curl -O curl -o aws-auth-cm.yaml https://amazon-eks.s3.us-west-2.amazonaws.com/cloudformation/2020-04-21/aws-auth-cm.yaml

Replace rolearn in the file (do not modify the file otherwise) with the correct value. To find this value:
- Open the AWS CloudFormation console.
- Locate and select the sourcegraph-worker-nodes row.
- Click the Output tab, and copy the NodeInstanceRole value.
Run kubectl apply -f aws-auth-cm.yaml
Watch kubectl get nodes --watch until all nodes appear with status Ready (this will take a few minutes).

Create the default storage class

EKS does not have a default Kubernetes storage class out of the box, but one is needed.

Follow these short steps to create it. (Simply copy and paste the suggested file and run all suggested kubectl commands. You do not need to modify the file.)

Deploy the Kubernetes Web UI Dashboard (optional)

See Tutorial: Deploy the Kubernetes Dashboard.

Deploy Sourcegraph! 🎉

Your Kubernetes cluster is now all set up and running!

Luckily, deploying Sourcegraph on your cluster is much easier and quicker than the above steps. :)

Follow our installation documentation to continue.