I’ve been following an excellent tutorialfor deploying a Docker image on an EC2 instance via GitLab CI/CD. It covers every step in the process in great detail. If you follow the steps then you’ll definitely end up with a working pipeline.
However, I still wasn’t quite sure how to handle the environment variables and credentials that I wanted to bake into the image, and which varied between my local development environment and the final deployed image.
This is how I got it to work with my existing GitLab Runner setup. I’m sure that there are more elegant ways to do this, but this is what worked for me.
I have a collection of environment variables which specify the attributes of the required database connection. I want to bake these into the Docker image. However, there are some constraints:
- I don’t want to include
.envin the Git repository. Because this is just a bad idea.
- The environment variables have different values in development and production.
- I don’t want a
.envfile lying around on the production server.
An acceptable solution would need to satisfy all of these requirements.
In my development environment the variables are specified in a
.env file which looks like this:
DB_HOST= DB_PORT= DB_USER= DB_PASSWORD= DB_DATABASE= DB_SCHEMA=
Server: GitLab CI/CD Variables
Since I wasn’t going to store these variables in the repository I needed some way to convey them to the deployed image. No problem: use GitLab CI/CD variables. There are two approaches to this:
- create variables in
- create variables in the GitLab UI.
I rejected the first of these for the reasons outlined above (don’t want to store variables in the repository itself).
So I created the following variables via the GitLab CI/CD UI:
ENV_FILE— (Type: File) The definition of the required environment variables using the same format as the
💡 This variable holds the contents of a file. It mocks the
.envfile that I have in my development environment.
SERVER_IP— (Type: Variable) The IP address for the server on which the Docker image is to be deployed.
SERVER_USER— (Type: Variable) The user on the server which will be used for the deployment.
SSH_PRIVATE_KEY— (Type: File) The SSH private key for the previously mentioned user.
🚨 If any of these variables are protected then the branch you are building and deploying from also needs to be protected.
This is what the definition of
ENV_FILE looks like (without the sensitive bits!):
Dockerfile needs to pick up the variables and stash them in the resulting image. And it needs to handle both of the scenarios above.
FROM python:3.8.6-slim # Rest of setup goes here... COPY .env .env
This is actually rather simple: it just copies the local
.env file onto the image. For the development environment this is easy (because there actually is a
.env file!). We need to do just a little more work in the CI/CD environment.
GitLab CI/CD Setup
In GitLab CI/CD the build process is specified by the
.gitlab-ci.yml file. This is what a very stripped down
.gitlab-ci.yml might look like:
variables: TAG_LATEST: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_NAME:latest TAG_COMMIT: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_NAME:$CI_COMMIT_SHORT_SHA build: image: docker:stable services: - docker:dind script: # Where is the environment file? - echo $ENV_FILE # File is outside of build context, so make local copy. - cp $ENV_FILE .env - docker build -t $TAG_COMMIT -t $TAG_LATEST . - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY - docker push $TAG_COMMIT - docker push $TAG_LATEST
Let’s rummage through some of the details.
TAG_COMMIT global variables which will be used to tag the resulting Docker image.
These tags are rather long and involved, but they will allow you to precisely locate the images in the GitLab image registry.
Image & Services
The build job will run in a
docker image using the
dind (Docker-in-Docker) service. To be clear, this means that Docker will be launching a container running Docker, hence “Docker in Docker”.
🚨 The GitLab runner will need to be configured to allow for Docker-in-Docker.
Take a look at the contents of the
It’s an absolute path pointing to a file containing the environment variables. This file is created dynamically by GitLab CI/CD using the value we assigned to the
ENV_FILE variable above.
The location of this file is not in the working folder, so we need to make a local copy in order for it to be included in the Docker build context.
cp $ENV_FILE .env
Once this is done, the rest is really just mechanics:
- build the image, assigning both of the tags
- login to the GitLab image registry and
- push the tagged images.
The resulting image can then be pulled from the GitLab image registry and run in the production environment, where it will use the environment variables specified in the GitLab CI/CD UI.