A blog icon.

Autoscaling Azure GitHub Runners — We Built Them So You Don’t Have To

Liatrio built an open-source solution for autoscaling self hosted GitHub Action Runners in Azure. We use Infrastructure as Code (IaC) using Terraform to automate Azure autoscaling runners.
A rube goldberg machine in muted color palette.

Enterprises will always need self-hosted solutions for deploying to private infrastructure, and that’s where we come in. Having had the opportunity to help several enterprise organizations implement best practices when it comes to building, deploying, and managing Infrastructure as Code, we saw an opportunity to give back to the community and create a self-hosted runner solution for GitHub Actions in Azure. Before getting into the details, you may find yourself wondering, “Why does this solution exist?”.

Some Background

First point and probably most important — enterprises need self-hosted solutions for deploying to private infrastructure. This is a bit redundant to the intro of this blog, but the importance of maintaining security of the runners that have access to your production environment is step zero when it comes to starting down the path of DevSecOps best practices.

Second, there are some awesome solutions such as actions-runner-controller that can run in your Kubernetes cluster regardless of cloud provider or on-prem and we LOVE this pattern. However, many build processes and anything that requires Docker in Docker (DinD) to run will open up several layers of vulnerability into your runner environment and has a higher barrier to entry to implement it securely. We believe running an autoscaling VM based solution in tandem addresses these concerns and provides a more comprehensive solution with a security-first approach.

Finally, we’ve seen and implemented many self-hosted runner solutions whether in Jenkins, Azure DevOps, or other popular CI/CD tools. Some of those solutions can be overwhelming in their initial setup. One decision we made when building this is that teams can use the images we provide early on and bring their own as they gain confidence that this is the best solution on the market. We’ll talk more about this in the next section as we go through the solution and some strong opinions we had when building it.

A Few Must Haves

Getting into the solution itself, we took some hard stances on certain things that must be adhered to in order for it to be a viable solution.

  1. Ephemeral only — runners should always be implemented in a way that they run one job and are destroyed, persisting data between runs not only opens your CI/CD pipelines to intermittent failures due to leftover data, it also could expose secrets or other sensitive information that introduces an attack vector that is very commonly exploited during attacks.
  2. Warm-pools are a must — teams that are developing, testing, and building code need a quick feedback loop and waiting several minutes for a runner to spin up is not acceptable. Maintaining a set of idle runners ready to serve requests on demand is a must if you want to enable your teams to move fast. Even though it’s not implemented yet, utilizing schedules to maintain how many runners are in the warm-pool is another benefit we hope to add to this solution soon™.
  3. Custom VM Images for fast feedback — developers want the tools required to run a build installed once and don’t want to reinvent the wheel every time they go to kickoff their CI workflow. Imagine coming into work everyday and someone deleted all the tools, packages, etc. that you use to do your job. That would be frustrating and so is having to do the same every time your CI is run. Utilizing custom images allows you to manage the VM image in code and InnerSourcing allows others within your company to contribute to the tools pre-installed.
A diagram of Liatrio's self-hosted Azure Github Actions Runner solution

Opening it Up to The Community

At this time, we expect this solution will see a lot of traffic and we’d love for any open source contributors to take part in making it even more awesome! We’re excited to see that great input that others have to add as this continues to grow. For those interested in contributing to the project, hop into our CONTRIBUTING.md and see how you can help. For those interested in using or getting help implementing this project, reach out to hello@liatrio.com and we’d love to partner with you in your DevOps journey!

Share This Article
Have a question or comment?
Contact uS

Related Posts

An illustration of a tree with a large trunk and numerous small branches, against a green gradient background.
GitOps: Defining the Best Infrastructure Pattern For You

A trunk-based GitOps approach enables users to deliver software more quickly and efficiently.

The IBM Cloud logo and AWS logo overtop of an abstract background.
IBM Cloud vs AWS: The Difference and How to Choose

In this post, we'll bring you a comparison between two of the main cloud providers: Amazon Web Services (AWS) and IBM Cloud.

Computers floating in the clouds in a Chris Ware art style.
Multi-Cloud vs Hybrid Cloud: The Difference and How to Choose

Multi-cloud vs hybrid cloud, what's the difference and how can you choose the best one for your situation?

An abstract digital blue background of cloud computing with the Azure and Terraform logos in the front.
We rebuilt Azure CAF in Terraform so you don’t have to!

In this introduction of Liatrio’s Terraform Azure CAF implementation we introduce the founding principles of our module, as well as some of the benefits it provides.

The Liatrio logo mark.
About Liatrio

Liatrio is a collaborative, end-to-end Enterprise Delivery Acceleration consulting firm that helps enterprises transform the way they work. We work as boots-on-the-ground change agents, helping our clients improve their development practices, react more quickly to market shifts, and get better at delivering value from conception to deployment.