Faster Feedback for Delivery Pipelines with Skaffold

Learn how Skaffold allows you to build, test, push and deploy applications on both the local developer’s laptop and the Jenkins pipeline.

The practice of continuous delivery seeks to increase the speed with which you’re able to deliver software changes to your users. The use of automation in building pipelines can cut the time required to build and deploy software. In addition, quality should be assessed throughout the delivery process to ensure software meets requirements before it’s delivered. In this post, we will look at how Skaffold, an open-source project built by Google, can be a valuable tool in the construction of continuous delivery pipelines for containerized workloads.

Challenges

First, let’s review the challenges that Skaffold can help address using a basic Spring Boot Java application. The application uses Maven for dependency management, compilation, unit testing and packaging of the application into a JAR file. The application also has a Jenkins Pipeline to orchestrate the continuous delivery of the application. As shown below, Maven is shared for both local development and the Jenkins pipeline.

Toolchain Without Skaffold

Although Maven provides some repeatability across the developer’s laptop and the Jenkins pipeline, this type of approach still poses many challenges:

  • Disparity of Tools - The version of tools (e.g., JDK, Maven, NodeJS, Ruby, Python) will likely differ between the developer laptop and the Jenkins environment. As a result, builds may succeed on a developer laptop but fail on the pipeline.
  • Disparity of Process - In the above example, the process for deploying on the developer’s laptop is to run mvn spring-boot:run, but the process for deploying on the Jenkins pipeline involves building a Docker image and deploying to a Kubernetes cluster with Helm. As a result, in some situations application behavior is inconsistent between the local development environment and the environment to which the pipeline deploys.
  • Commit/Push Cycle - In order for the developer to test changes to the Jenkins pipeline, the developer must create a new commit and push the commit to their source code management system to execute the test.
Disparity between the developer’s laptop and the pipeline negatively impacts feedback time.

In all cases above, it takes a significant amount of time for a developer to get feedback on a change. Developers must troubleshoot disparities between the environments by either aligning their local environment to match that of the Jenkins pipeline or making “blind commits” in which the developer makes a change and pushes just to see if it works on the Jenkins environment. This delay in feedback time also impacts the overall lead time of the delivery pipeline.

The Solution

Skaffold offers easy, repeatable Kubernetes development. It's a command line tool that allows you to define and run the workflow for building, pushing and deploying your application. As shown below, leveraging Skaffold on the developer’s laptop and in the Jenkins pipeline results in a consistent and repeatable delivery process.

Toolchain with Skaffold

In this model, the developer is able to get fast feedback on the delivery pipeline by running Skaffold locally. The developer also has a much higher level of confidence that any changes will function the same on the Jenkins pipeline as they do locally, decreasing the level of rework.

Skaffold in Action

The definition of the workflow used exists in a file named skaffold.yaml. This file is stored in the application source repository. The file provides the necessary instructions that allow Skaffold to build a Docker image, test the image, tag the image, push the image to a repository and deploy the image via Helm to a Kubernetes cluster.

apiVersion: skaffold/v1beta3
kind: Config
build:
artifacts:
- image: knowledge-share-app
test:
- image: knowledge-share-app
structureTests:
- ./image-tests/*
deploy:
helm:
releases:
- name: knowledge-share-app
chartPath: charts/knowledge-share-app
values:
image: knowledge-share-app

Skaffold defers to Docker to perform all build activities. In this case, we use multi-stage builds in which the first stage uses a base image for Maven to perform the compilation, unit testing and packaging of the JAR file. The second stage then uses a smaller base image with OpenJDK to run the JAR file. This approach ensures that the versions of the tools (e.g., Maven and JDK) are exactly aligned between the local development environment and the Jenkins pipeline.

FROM maven:3-jdk-8 as builder
EXPOSE 8080

USER root
RUN apt-get update && mkdir /app
WORKDIR /app

# Keep maven dependencies in separate image layer
COPY pom.xml .
RUN mvn dependency:resolve dependency:resolve-plugins -B

# Copy in the src and build
COPY src ./src
RUN mvn clean package -B

FROM openjdk:8-jre-alpine
WORKDIR /app

COPY --from=builder /app/target/knowledge-share-app.jar .
ENTRYPOINT ["java","-jar","knowledge-share-app.jar"]
EXPOSE 8080

For local development, the developer runs skaffold dev to start the process locally, which then monitors the source code and continuously runs the build, tag, push, deploy process so the developer can get fast feedback on any changes made.

All unit tests are executed as a part of the building of docker image. In addition, Skaffold uses the Container Structure Test framework to perform static testing of the image. As shown below in a sample test file, the framework can validate the image that was created before proceeding to the deployment step.

schemaVersion: "2.0.0"

commandTests:
- name: "java exists"
command: "java"
args: ["-version"]
expectedError: ["OpenJDK Runtime Environment","1.8.0_201"]
exitCode: 0
fileExistenceTests:
- name: 'knowledge-share-app.jar'
path: '/app/knowledge-share-app.jar'
shouldExist: true
metadataTest
exposedPorts: ["8080"]
entrypoint: ["java","-jar","knowledge-share-app.jar"]
workdir: "/app"

Let’s see Skaffold in action! The screencast below shows running skaffold dev locally, making a change, running the test automation and deploying the test … all in under 3 minutes!

skaffold dev

Now let’s look at how we incorporate Skaffold into our Jenkins pipeline. We call skaffold build from our Jenkinsfile to build, test and push our image to our Docker registry. We use skaffold build instead of skaffold dev because we want Jenkins to perform a single build and push of the image rather than continuously monitoring, building and deploying the code.

pipeline {
agent {
label "builder-images"
}
environment {
SKAFFOLD_DEFAULT_REPO = 'docker.artifactory.liatr.io/liatrio'
}
stages {
stage('Build') {
steps {
// Create sonar.properties for sonar maven plugin
withCredentials([string(credentialsId: 'sonarqube', variable: 'sonarqubeToken')]) {
sh "echo 'sonar.login=${sonarqubeToken}' >> sonar.properties"
}
// Create and test image with skaffold
container('skaffold') {
script {
docker.withRegistry("https://${SKAFFOLD_DEFAULT_REPO}", 'artifactory-credentials') {
sh "skaffold build"
}
}
}

}
}
}
}

Learn More

Skaffold allows you to consistently execute the workflow necessary to build, test, push and deploy an application on both the local developer’s laptop and the Jenkins pipeline. This parity of tooling reduces rework through use of a consistent process while also improving the feedback time for changes that the developer makes to applications, ultimately improving the overall lead time of the delivery pipeline.

Want to learn more about Skaffold? Reach out! Also check out these resources:

Have a question or comment?
Contact uS
About Liatrio

Liatrio is a collaborative, end-to-end Enterprise Delivery Acceleration consulting firm that helps enterprises transform the way they work. We work as boots-on-the-ground change agents, helping our clients improve their development practices, react more quickly to market shifts, and get better at delivering value from conception to deployment.

Ready to Accelerate Delivery and Transform Your Organization?

Contact Us
»