How Much Will K8s GPU Tinkering Cost? (AWS Edition)

How much will it cost me to run casual experiments with AWS, Kubernetes and GPU instance topics?

Let's see how to learn more about AWS pricing, instance options and useful services which can reduce cloud costs for GPU and Kubernetes experiments.

The Region Choice Matters

The price varies by region (here's a cool article, looking into the topic of region-specific price variations for different non-GPU instance types). The takeaway seems to be, that N. Virginia and Ohio are the least expensive regions, at least for the selected non-GPU instance types at the type of writing.

On-Demand GPU Instance Prices

You can view the current GPU instance prices per hour on the official AWS pricing page. Just scroll down to "GPU Instances - Current Generation".

At the time of writing, the cheapest GPU instance type in US East seems to be g4dn.xlarge with $0.526 per hour. It has 4 cores and 16 GiB of RAM. A comparable non-GPU instance t3.xlarge costs $0.1664 per hour.

If you need more power and are ready to pay a bit more, p2.xlarge, g4dn.2xlarge and g3s.xlarge are the next-more-expensive choices. Unfortunately, the table does not provide much insight into what GPUs are available with those instances.

Note: as AWS bills per second, you can spend less than a fixed one-hour increment price.

What GPUs Do They Offer?

There's an overview of available GPUs per instance type can be found here, in the docs of the Deep Learning AMI developer guide.

It seems like G4 have NVIDIA T4 GPUs, while P2 have NVIDIA K80 GPUs and G3 have NVIDIA Tesla M60 GPUs.

What About Spot Instance Prices?

You can find them here. At the time of writing, the g4dn.xlarge instance type from above would cost $0.1578 per hour. That's only one third of the on-demand pricing, and a great deal if you don't care that your instance might go away at any time.

When it comes to experimenting with cloud GPUs and Kubernetes, this is an easy optimization to make.

Managed Kubernetes Services

For running a managed Kubernetes cluster, AWS offers Amazon EKS. You pay $0.10 per hour for each cluster, in addition to the resources you use (like running instances).

The Bottom Line

So it seems that, if I wanted to experiment with Kubernetes and GPU instances on AWS, it wouldn't cost that much at all. About $1 per hour of active work seems to be a reasonable estimate. After all, it should be easy to spin up a fresh cluster with terraform when it's needed, and destroy it after a successful session. (As long as provisioning times won't be too long - time is a scarce resource after all.)

I don't need to stick to any particular region, because latency won't be much of an issue, and can go with the cheapest ones.

Depending on the convenience of using spot instances, that price can go down even more, but given the sporadic and time-efficient nature of such experimenting, I don't see a point in doing much about price optimizations for the sake of saving a few cents.

Of course, if you are planning on having a semi-permanently Kubernetes cluster with GPU instances running on AWS, there's a lot more sense in optimizing for a lower price if possible. But that's a topic for a future article.