As technology continues to evolve, one thing has stayed the same: customer demand for rapid change. To keep up with this demand, DevOps teams are working to simplify the continuous deployment process. In this post, I’ll review a new(ish) term called GitOps and look at how you can integrate a tool like Flux, an open-source project by Weaveworks, into your continuous delivery pipeline to automate the deployment of your application code more frequently and consistently.
GitOps
Managing the deployment of applications while also producing high-quality code can be tough. For this reason, large enterprises often create roles for team members, much like the roles on a football team. For instance, I can think of the quarterback as the developers who produce code for the receivers (QA testers), who set up the code for the placekicker (deployment manager) to kick the extra point. If these team members don’t work together well, they usually end up losing. The business side of things isn’t all that different. If the team doesn’t work well together, they usually have long release cycles or worse, failed releases.
GitOps simplifies deployment by automating most of it and keeping the deployment definitions within your version control system. Streamlining the deployment process allows team members to choose their positions on the team. GitOps uses your Git source management system to define deployment rules and delivery options to automatically deploy based on commits or pull requests to your source code repository. You can then use Flux to monitor your version control systems and clusters to determine when updates need to be rolled out. Flux also allows previous versions of applications to be easily rolled back since the commit history holds all versions.
GitOps does the following:
- Allows a single repo (source of truth) to control deployments to your infrastructure.
- Provides the benefits of version control stored with your source code.
- Helps to ensure all changes are rolled out and releases aren’t forgotten.
- Facilitates Weaveworks Flux adoption because the team uses the Git system they may already know.
Why Weaveworks Flux? Flux is a unique tool with a fairly limited scope. Now that Flux is working with Inuit and Amazon on Argo Flux, Flux is THE tool for GitOps on Kubernetes. If you need a tool for GitOps on virtual machines or something besides Kubernetes, Flux might not be for you. But if you're only looking to do GitOps with Kubernetes, it’s exactly what you need.
—Jon Rudy
Weaveworks Flux
Developed by Weaveworks, Flux is an open source CD tool that simplifies GitOps deployments. Flux helps automate the deployment of your helm releases by automating the rollout of newly created containers/images as they’re built. It actively communicates with a team’s version control system and Kubernetes cluster to monitor images that are outdated or out of sync with the latest images in the system and then rolls out the most recent changes to keep your cluster up to date.
I like to think of Flux as my mother when I was growing up. She always made sure I looked my best before heading out the door to school every morning, similar to the way Flux ensures your latest and greatest deployments are rolled out. Flux takes advantage of an operator in the cluster to trigger Kubernetes deployments, meaning you don’t need to add another CD tool into the cluster. Although Flux can be deployed on its own, it’s most valuable
when used at the end of a CD pipeline, ensuring the latest successfully built images are deployed to your cluster.
Weaveworks Flux benefits include the following:
- Automated Git to Cluster Synchronization: Any changes made to your git repository can be automatically rolled out to your cluster without having to deploy anything yourself.
- Automated Deployment of New Images: Flux continuously monitors version control systems to detect newly created images and automates the rollout of images as they’re created, ensuring the latest changes successfully make it into the cluster.
- Integration with Other DevOps Tools: Flux easily integrates with DevOps tools such as Helm, Kustomize, and Flagger.
- Ease of Setup: It’s easy and straightforward to deploy into a cluster, and little configuration is needed to hook up with helm releases.
- Active/Open Source Repository: This new tool is being improved on an ongoing basis.
Flux’s Helm Operator
Flux can be used to deploy or update your helm releases in your cluster by deploying a helm operator to watch for changes to custom resources of type HelmRelease. When a HelmRelease is seen in the cluster, the operator installs or updates that release depending on its current state. The sample HelmRelease schema below shows how to construct one for an application.
Metadata includes information such as the name of the release, the namespace to deploy it into, and annotations to help Flux understand what to do with the HelmRelease. Here, the annotations tell Flux to automate the rollout of new images and update the chart-image tag as long as it matches the defined regex.
Spec is where you cite the helm release name, along with the chart you want to update or apply. In this case, let’s point to a chart in the git repository itself.
Values are the values in the helm chart that will be updated. In this case, we’re pointing to the Artifactory repository and specifying a tag for the image we want to deploy. When Flux detects a newly created image in the repository, it updates the tag automatically in the source control system and also rolls out the image in the cluster.
Flux Benefits
With so many deployment tools and options available, you may be wondering why you should give Flux a shot. If you or your clients are currently deploying to Kubernetes, here are three reasons you should consider using Flux:
- Security: Flux operates directly inside the cluster, which helps eliminate possible attacks that require work outside of a cluster, such as having to store and pass credentials into other CD systems that may require them to deploy. If helm 2 is still being used inside your cluster, Flux can be more securely locked down to have only the permissions of those RBAC permissions configured inside your cluster. Flux can also connect with the deployed tiller service account to provide tls encrypted traffic.
- Configuration as Code: Flux can be configured directly inside your source code repository. Because a single source of Git can control both your applications and the deployment of your code in multiple environments, teams don’t have to manage multiple repositories. Once a deployment is configured, your application will also be continuously deployed so you don’t miss releases and pipelines don’t fail your deployments.
- Easy Adoption: One reason a lot of new tools are shot down is the difficulty of adoption. With Flux, adoption is simple! Many engineers already have experience with Git, which is the central control system for Flux. Many GitOps tools can be easily modified to integrate with Flux, such as Jenkins, Helm, and Skaffold. Finally, Flux is easy for developers to manage. Developers don’t have to deal with complex CI/CD pipelines built around their applications in order to deploy their code. The deployment configuration sits alongside source code, greatly simplifying the deployment process. Automation that Flux handles to continuously deploy the latest changes also gives engineers more time to develop their code and reduces worries about how to deploy code.
Weaveworks Flux in Action
Let's jump into a demo of how Flux can be used to continuously deploy a node js application being deployed by a helm chart. (For any related source code below, check out the Github repo.)
Tools being used:
- Docker for Desktop - (Local Kubernetes Cluster).
- Github - Where the application, the helm chart for the application, the HelmRelease Flux custom resource definition, and the Jenkinsfile to define a job are stored.
- Helm - Used to perform a helm release with the application defined inside a helm chart.
- Flux - Used alongside the helm-operator to continuously deploy the application.
- Skaffold - Used to build the Docker image and tag it to deploy to Artifactory.
- Jenkins - Used to run a job to build and deploy the image.
- Artifactory - Used to store the application images as they are created.
I’ve created a diagram below which shows the role each tool plays. Basically, a developer commits changes to Github, which then triggers a Jenkins job that’s polling for changes. The Jenkins job then runs and builds a new image for the application and deploys that image to Artifactory. Once this image is in Artifactory, Flux notices the new tag and takes care of the continuous deployment by updating the source code and deploying the latest changes into the cluster.
Follow along using the steps below:
Step 1: Create the HelmRelease Custom Resource for Flux. This configuration will define the deployment for the helm chart and tell Flux exactly what we want it to do. The most important pieces are the annotations, chart, and image repository. The annotations configure Flux to automate the deployment of the images matching the regex, the chart points to the node application helm chart, and the image repository points to the version control system to monitor for changes. (For additional resources, visit flux_release.yaml.)
Step 2: Deploy Flux into the Cluster. First, add the Flux repository to the list of helm repos.
Second, create the HelmRelease CRD definition.
Third, take advantage of helm and Flux’s helm chart to install Flux directly into the cluster. (The git.url should point to your repo.)
Fourth, add a deploy key to the repository Flux will be updating. Begin by running the command below to get the key pair with which to create the deployment key.
Finally, head over to your repository on Github, and in the settings paste the contents into a new key with write access.
To see your application deployed, run the command below.
Step 3: Create the Jenkins Job to Scan the Repository. On your Jenkins server, create a new pipeline job that will point to your repo with scm polling. Here’s a sample:
In addition, a simple Jenkinsfile should be created in the sample repo that runs skaffold to build and deploy the job to Artifactory. (Note: You’ll need an agent that has skaffold installed to take advantage of skaffold commands in your pipeline. However, this setup isn’t necessary, as the only requirement is to have a Jenkinsfile that builds and deploys images to Artifactory.) You can find the referenced skaffold agent on github, along with other builder images. (For additional resources, see Jenkinsfile and skaffold.yaml.)
Step 4: Make a Change to the Application to Trigger a Build. First, port-forward the application to look at its current state and make a simple change.
Next, change the background color to the application. Edit the server.js file and edit the background-color on line 12.
Finally, push up the changes to kick off a Jenkins job.
Once the job has finished running on Jenkins, check the logs to verify that a new version of the application was tagged and deployed. You should see a message in the logs similar to the one below.
Step 5: Verify the Image on Artifactory. Browse the Artifactory instance and match the tag to a newly created image.
Step 6: Let Flux Handle the Continuous Deployment of the Application. Flux should now detect this latest image and create a new commit to the repo to update the HelmRelease version. It should also automatically deploy the image by updating the HelmRelease.
First, verify the newly created commit by Flux, which automates the process of updating the image tag for your deployment.
Second, verify that the helm release inside the cluster was updated.
Finally, verify that the background was actually updated by port-forwarding the service.
The Flux of the Future: Argo Flux
A tool like Weaveworks Flux can help automate the delivery of your application source code, and the exercise above is a simple example of how you might incorporate Flux into your Kubernetes deployment pipelines. As the exercise shows, I can easily incorporate Flux directly into the cluster, and the operator can detect new images of the application as they’re built, ensuring those images are deployed as automated changes to the source code are made.
A couple of months ago, I heard some exciting news about Argo Flux. Intuit, creator of the popular open source project Argo CD, and Weaveworks Flux have joined forces to deliver Argo Flux, a single tool chain for continuous deployment and automated workflows using GitOps.
Be sure to stay up to date with upcoming Argo Flux development and expected features. You can also check out some of the resources below:
Also feel free to reach out with questions and comments!
