Using Serverless Secrets in Jenkinsfile Pipelines

In this tutorial, I’ll explain how to use Serverless secrets in Jenkinsfile pipelines. In particular, I’ll explain how to use secrets as environment variables in Serverless applications, store those credentials in Jenkins, and have a Jenkinsfile pipeline deploy a serverless application using Docker that references those newly defined variables.

The example application used to explain these concepts will be the serverless-slack-app1 by John Agan on GitHub. This blog assumes introductory level knowledge of Git, Docker, Serverless, Jenkins, and Jenkinsfile pipelines.

To summarize, here are the goals of this Serverless secrets in Jenkinsfile pipelines tutorial:

1. Create a basic Jenkinsfile with Docker agents that deploys a Serverless application.
2. Convert custom variables in serverless.yml to reference environment variables.
3. Store Serverless secrets in Jenkinfile pipelines.

Step 1: Serverless Jenkinsfile Using Docker

Example Application

If you want to DIY as opposed to just reading, feel free to start with the example application. Keep in mind you’ll need a Jenkins instance with Docker available and an Amazon Web Services account that may incur charges for deploying the serverless application. If you want to spin up a Jenkins container that has Docker and Jenkinsfile pipelines already configured, feel free to use Liatrio’s Alpine-Jenkins image.

$ npm install -g serverless
$ serverless install --url https://github.com/johnagan/serverless-slack-app
$ cd serverless-slack-app/

Agent

For this Serverless application, we’re just going to use one Docker image as the agent. Since we want to keep it lightweight, we’re going with the Alpine version of Node, which comes in at roughly 71.1MB. (If you aren’t using Docker with Jenkins you can refer to the example on the Jenkins CI GitHub wiki that doesn’t use Docker.)

We’re also going to need to be running as root in the container to properly run Serverless commands, so we’ll specify running as root in the arguments.

Since we’re only using one image throughout the whole pipeline, we can define it just once in the beginning of the pipeline.

 agent {
 	docker {
  	image 'node:alpine'
    args '-u 0:0'
  }
}

‍Environment

For the environment of the pipeline, we’re going to reference AWS credentials from Jenkins. For details on how to add credentials to Jenkins, refer to the Adding Docker Hub Credentials to Jenkins section from Building with Docker in Jenkins Pipelines. You can also refer to the official documentation.

environment {
	AWS_ACCESS_KEY_ID = credentials('aws-secret-key-id')
  AWS_SECRET_ACCESS_KEY = credentials('aws-secret-access-key')
}

These environment variables will be referenced automatically when the Serverless application is deployed. However, they will only be referenced automatically if a profile is not defined in serverless.yml. So, go ahead and delete this line in the example application’s serverless.yml file.

profile: serverless

With this line removed, the Serverless deployment will search for the default AWS profile, which is what we have defined in the Jenkinsfile.

Simple Serverless Jenkinsfile Using Docker

To keep things simple, we’re only going to define a Build stage and a Deploy stage for the Jenkinsfile. However, a Test stage could easily be added that ran an npm testcommand. Here’s the resulting Jenkinsfile that will successfully build and deploy our example application.

pipeline {
	agent {
  	docker {
    	image 'node:alpine'
      args '-u 0:0'
    }
  }
  environment {
  	AWS_ACCESS_KEY_ID = credentials('aws-secret-key-id')
    AWS_SECRET_ACCESS_KEY = credentials('aws-secret-access-key')
  }
  stages {
  	stage('Build') {
    	steps {
      	sh 'npm install'
      }
    }
    stage('Deploy') {
    	steps {
      	sh 'npm install -g serverless'
        sh 'serverless deploy'
      }
    }
  }
}

This deploys our Serverless application! However, this works only if the Slack secrets are defined as text in the serverless.yml file. Let’s fix that.

Step 2: Set Custom Serverless Variables

Reference Environment Variables in Serverless Configuration

As it stands, our example application has required Slack variables defined as custom variables in serverless.yml. We need to change that so the variables are obtained from the environment. Luckily, the Serverless framework has a way to do that outlined in their official docs.

Before, we had something like this where the x’s represented secrets.

  slack_verification_token: "xxxxxxxxxxxxx"  
  ...
  slack_client_id: "xxxxxxxxxx.xxxxxxxxxxxx"
  slack_client_secret: "xxxxxxxxxxxxxxxxxx"

We can convert these into environment variable references.

slack_verification_token: ${env:SLACK_VERIFICATION_TOKEN}
...
slack_client_id: ${env:SLACK_CLIENT_ID}
slack_client_secret: ${env:SLACK_CLIENT_SECRET}

Great! Now our secrets are never defined in code.

Add Credentials to Jenkins

We need to add the SLACK_VERIFICATION_TOKEN, SLACK_CLIENT_ID, and SLACK_CLIENT_SECRET variables as secret text variables in Jenkins so that we can reference them in our Jenkinsfile. If you’re unsure how to do this, refer to the Environment section above.

Step 3: Finish Creating the Environment for Serverless Secrets in Jenkinsfile Pipelines

Now we need to add the newly defined credentials to our Jenkinsfile pipeline environment. We can do this the same way we defined our AWS credentials.

environment {
	SLACK_VERIFICATION_TOKEN = credentials('slack-verification-token')
  SLACK_CLIENT_ID = credentials('slack-client-id')
  SLACK_CLIENT_SECRET = credentials('slack-client-secret')
  AWS_ACCESS_KEY_ID = credentials('aws-secret-key-id')
  AWS_SECRET_ACCESS_KEY = credentials('aws-secret-access-key')
}

Serverless Secrets in Jenkinsfile Pipelines

We’ve successfully created a Jenkinsfile that automatically deploys a Serverless application using custom secrets stored in Jenkins. If you want to see all of the code implemented in this Serverless secrets in Jenkinsfile pipelines tutorial (including the complete serverless.yml and Jenkinsfile), please check out the GitHub repository.

If you run into any issues, leave a comment below or tweet us @liatrio.