How to optimize cost with AWS Graviton and Spot in Amazon Elastic Kubernetes Service (EKS)

The goal of this article is to show how to set up a multi-arch EKS cluster and use spot managed group. Indeed diversification across multiple instance types are important with Spot so why not adding Graviton2 based instances? The application is a hello-world web application developed in Java, in Go and NodeJS.

The code is available in EKS-multi-arch.

So the following steps will be done:

• Install all the needed tool in an instance
• Create an EKS cluster
• Create an ECR repository
• Add a spot ARM managed node group
• Create multi-arch Docker images for Java/Go/NodeJS application
• Install a deployment for all the created images into EKS
• Verify if pods work well in amd64 and arm64 arch.

AWS Graviton

Just a quick reminder about AWS Graviton.

AWS Graviton Processors are custom built by Amazon Web Services using 64-bit Arm Neoverse cores to deliver the best price-performance for your cloud workloads running in Amazon EC2. AWS Graviton2-based deliver up to 40% better price-performance over comparable current-generation x86-based instances1 for a broad spectrum of workloads such as application servers, microservices, video encoding, high-performance computing, electronic design automation, compression, gaming, open-source databases, in-memory caches, and CPU-based machine learning inference.

Amazon EC2 Spot

Amazon EC2 Spot Instances let you take advantage of unused EC2 capacity in the AWS cloud. Spot Instances are available at up to a 90% discount compared to On-Demand prices.

Install tools

I use a Linux 2 AMI for this article so some commands can change if you use a different distribution or your laptop.

Install git, docker, eksctl, kubectl, docker-buildx

sudo yum update -y

# install AWS CLI (already installed in Amazon Linux 2)
# https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html


# install git
# https://github.com/git-guides/install-git
sudo yum install git -y

# install docker
# https://docs.docker.com/get-docker/
sudo amazon-linux-extras install docker -y
sudo service docker start
sudo usermod -a -G docker $USER
# You must logout and login or restart to let the system run group policy again and add the current user to the docker group

# verify if everything works
docker info

# This package does not contain the buildx command. So I need to install it manually.
# https://github.com/docker/buildx#installing

# Buildx is still experimental
export DOCKER_CLI_EXPERIMENTAL=enabled

mkdir -p ~/.docker/cli-plugins/
wget -O ~/.docker/cli-plugins/docker-buildx https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.linux-amd64 

chmod a+x ~/.docker/cli-plugins/docker-buildx

# Install Kubectl
# https://docs.aws.amazon.com/eks/latest/userguide/install-kubectl.html
curl -o kubectl https://amazon-eks.s3.us-west-2.amazonaws.com/1.18.9/2020-11-02/bin/linux/amd64/kubectl
chmod +x ./kubectl
sudo mv ./kubectl /usr/local/bin

kubectl version

# Install eksctl
# https://docs.aws.amazon.com/eks/latest/userguide/eksctl.html

curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
sudo mv /tmp/eksctl /usr/local/bin
eksctl version
# clone the repository
git clone https://github.com/florent-brosse/EKS-multiarch.git
cd EKS-multiarch

Let’s create an env variable for the region and the account

export REGION=eu-west-1
export ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)

Create a repository in Amazon Elastic Container Registry (ECR)

To retrieve a password to authenticate to my registry

aws ecr get-login-password --region $REGION | docker login --username AWS --password-stdin $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com

Create a repository to push all my hello-world application. I will put different hello word implementation done in different language into the same repository but different tags.

aws ecr create-repository \
    --repository-name hello-world \
    --image-scanning-configuration scanOnPush=true \
    --region $REGION

This image can be used to install emulators for architectures your node does not have native support so that you can run and build containers for any architecture.

docker run --privileged --rm tonistiigi/binfmt --install all

I can create new instances using the docker buildx create command. This creates a new builder instance with a single node based on my current configuration.

docker buildx create --name builder --use
docker buildx inspect --bootstrap

You should see at least in Platforms: linux/amd64 and linux/arm64.

Create the EKS cluster

Create an EKS cluster with an on-demand x86 managed node group.

eksctl create cluster --name=eks-arch-managed-node-groups --instance-types=m5.xlarge,m5a.xlarge,m5d.xlarge --managed --nodes-max=5 --nodes-min=1 --nodes=1 --asg-access --nodegroup-name on-demand-amd-4vcpu-16gb --region=$REGION

Add a spot ARM managed node group

eksctl create nodegroup --cluster eks-arch-managed-node-groups --instance-types m6gd.xlarge,m6g.xlarge --managed --spot --name spot-4vcpu-16gb --asg-access --nodes-max=5 --nodes-min=1 --nodes=1 --region=$REGION

Docker and multi-arch image

Building multi-architecture Docker images is still an experimental feature. However, hosting multi-architecture images is already well supported by Docker’s Registry.

docker buildx imagetools inspect ubuntu:21.04

The result is:

Name:      docker.io/library/ubuntu:21.04
MediaType: application/vnd.docker.distribution.manifest.list.v2+json
Digest:    sha256:b6dc45a852dc83fa0e7504e9d68b9b0084eefb8aeb5f295f276bf99f5c033490

Manifests:
  Name:      docker.io/library/ubuntu:21.04@sha256:eb9086d472747453ad2d5cfa10f80986d9b0afb9ae9c4256fe2887b029566d06
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/amd64

  Name:      docker.io/library/ubuntu:21.04@sha256:017b74c5d97855021c7bde7e0d5ecd31bd78cad301dc7c701bb99ae2ea903857
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/arm/v7

  Name:      docker.io/library/ubuntu:21.04@sha256:bb48336f1dd075aa11f9e819fbaa642208d7d92b7ebe38cb202b0187e1df8ed4
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/arm64/v8

  Name:      docker.io/library/ubuntu:21.04@sha256:29c2f09290253a0883690761f411cbe5195cd65a4f23ff40bf66d7586d72ebb7
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/ppc64le

  Name:      docker.io/library/ubuntu:21.04@sha256:e8e0c3580fc5948141d8f60c062e0640d4c7e02d10877a19a433573555eda25b
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/s390x

We can see that the image is built for different architectures. For more information: https://docs.docker.com/registry/spec/manifest-v2-2/

Docker manages to choose the right version they should pull and run according to the architecture of the docker engine. So you can use the same command docker run -t -i --rm ubuntu bash in amd64 or arm64.

Let’s do a hello world web app and push multi-architecture docker images into ECR.

I’ve taken Java for the compiled into bytecode language, for the interpreted language I’ve chosen NodeJS and for the compiled language Go. So with these examples, we have all types of language. For example, Python should use the same way than NodeJS because it’s also an interpreted language.

JAVA

For Java, the code is compiled to Java bytecode and it’s architecture-independent. So when you build in an arm or amd64 the Java Bytecode will be the same. In this example, I will build the package into docker using a multistage build to avoid installing Java and maven locally. I use FROM --platform=$BUILDPLATFORM maven:3-openjdk-15-slim AS build to avoid building the image two times. Indeed the second times the build layer will be cached.

cd java

Create the Docker image for amd64 and arm64 architecture and push it to ECR

docker buildx build --progress plain --platform linux/amd64,linux/arm64 -t $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com/hello-world:java --push .

Verify if the images of hello-world:java are available in ECR

docker buildx imagetools inspect $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com/hello-world:java

I’ve got another version of a Dockerfile without the multi-stage part. I need to compile my application into Java bytecodes before building the docker image with the DockerfileNoStage file. So let's install a JDK first.

# Install a JDK 15
# https://docs.aws.amazon.com/corretto/latest/corretto-15-ug/what-is-corretto-15.html
# or 
# https://adoptopenjdk.net/installation.html
sudo rpm --import https://yum.corretto.aws/corretto.key
sudo curl -L -o /etc/yum.repos.d/corretto.repo https://yum.corretto.aws/corretto.repo
sudo yum install -y java-15-amazon-corretto-devel

java --version

compile and package the app with the maven wrapper tool

./mvnw clean package

Create the Docker image for amd64 and arm64 architecture and push it to ECR

docker buildx build --progress plain --platform linux/amd64,linux/arm64 -t $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com/hello-world:javaNoStage -f DockerfileNoStage --push .

Verify if the images of hello-world:javaNoStage are available in ECR

docker buildx imagetools inspect $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com/hello-world:javaNoStage

Go

cd ../golang

Go need to be built for different architectures. I use a multi-stage Dockerfile, to get the right building image during the build process.

Create the Docker image for amd64 and arm64 architecture and push it to ECR

docker buildx build --progress plain --platform linux/amd64,linux/arm64 -t $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com/hello-world:go --push .

Verify if the images of hello-world:go are available in ECR

docker buildx imagetools inspect $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com/hello-world:go

Go is also able to build an application for different Operating Systems and architectures by using environment variables. So I have another Dockerfile version witch used the application already built locally.

# Install golang
sudo yum install golang -y
# Create the go application for arm64 architecture and name it 'arm64'.
CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -a -installsuffix cgo -o arm64 .
# Create the go application for amd64 architecture and name it 'amd64'.
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -installsuffix cgo -o amd64 .

Create the Docker image for amd64 and arm64 architecture and push it to ECR

docker buildx build --progress plain --platform linux/amd64,linux/arm64 -t $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com/hello-world:goNoStage -f DockerfileNoStage --push .

Verify if the images of hello-world:goNoStage are available in ECR

docker buildx imagetools inspect $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com/hello-world:goNoStage

There is another version of the Dockerfile for the Go application DockerfileBuildSamePlatform. This version avoids pulling another version of the build image.

NodeJS

cd ../nodejs

For NodeJS, the application doesn’t to be build or package.

Create the Docker image for amd64 and arm64 architecture and push it to ECR

docker buildx build --progress plain --platform linux/amd64,linux/arm64 -t $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com/hello-world:nodejs --push .

Verify if the images are available in ECR

docker buildx imagetools inspect $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com/hello-world:nodejs

Deploy the applications into EKS

Install gettext to use ensubst to replace env variable in the YAML deployment file.

yum install gettext
cd ..
envsubst < go.yaml | kubectl apply -f -

envsubst < java.yaml | kubectl apply -f -

envsubst < javaNoStage.yaml | kubectl apply -f -

envsubst < nodejs.yaml | kubectl apply -f -

View the pods

kubectl get pod -o wide

The deployments use both nodes.

kubectl get nodes --show-labels | grep arch

One node is amd64 and the other arm64.

To try a specific application and pod you can use the current IP of a pod inside a container with the image curlimages/curl:

kubectl run -it --rm curlimage --image=curlimages/curl -- sh

curl http://${ip-of-a-pod}:${app-port}

You can also get the logs:

kubectl logs -l app=nodejs
kubectl logs -l app=golang
kubectl logs -l app=java
kubectl logs -l app=java-no-stage

The Java, Go and NodeJS applications work well in amd64 and arm64.

Clean up resources

eksctl delete cluster --name=eks-arch-managed-node-groups --region=$REGION
aws ecr delete-repository \
    --repository-name hello-world \
    --region $REGION \
    --force

Conclusion

In this demo, we created multi-arch Docker images and use them with EKS. By combining Graviton 2 and spot, the potential saving is very important. For this demo, I used 2 different managed node group for ARM and x86 to be sure to deploy my pod into the 2 architectures but the different architecture could be into the same managed node group.