Install GitLab Runner with Docker

DockerCIGitLab / 5 min read

📢 An updated version of this post reflecting recent changes in GitLab Runner can be found here.

I’ve got a project which takes a long time to build. And I rebuild it regularly. I’ve been using the shared runners on GitLab. However, the total time constraint has become a limitation. I’m going to install GitLab Runner as a Docker service on an underutilised EC2 instance.

Background information on Continuous Integration with GitLab can be found here.

Install Docker

Install Docker.

sudo apt update
sudo apt install -y docker.io

Add ubuntu user to the docker group.

sudo gpasswd -a ubuntu docker

Login again.

Start Runner Container

First create an instance of the GitLab Runner container.

docker run -d --name gitlab-runner --restart always \
    -v /srv/gitlab-runner/config:/etc/gitlab-runner \
    -v /var/run/docker.sock:/var/run/docker.sock \
    gitlab/gitlab-runner:v15.8.2

I’ve specified a particular version of the image (v15.8.2), but if you want to live on the edge then you could grab the latest version instead.

More information about the GitLab Runner can be found here.

Obtain a Token

Either for a single project or a group:

  • go to Settings > CI/CD
  • expand the Runners section
  • copy the token from the Set up a specific Runner manually section.

Register a Runner

To register a runner, use the register command.

docker exec -it gitlab-runner gitlab-runner register

💡 This seems repetitive, but there’s a reason for the repetition: the first gitlab-runner refers to the running container, while the second reference specifies a command to run within the container.

📌 If you need to use Docker-in-Docker (for example, to build a Docker image as part of the CI/CD process) then you should also specify the --docker-privileged flag.

docker exec -it gitlab-runner gitlab-runner register --docker-privileged

Provide the following input:

  • Specify https://gitlab.com/ as the coordinator URL.
  • Provide the token.
  • Provide a description for the runner.
  • Supply tags for the runner (can leave this blank for the moment).
  • Specify docker as the executor.
  • Specify a default Docker image (for example, rocker/verse:latest) which will be used if an image is not given in .gitlab-ci.yml.

To wrap things up you can execute the run command, which will run the most recently added runner and echo its logs to the terminal.

docker exec -it gitlab-runner gitlab-runner run

The runner should be automatically restarted. However, you can also restart manually just to be sure:

docker restart gitlab-runner

If you refresh the Settings > CI/CD page you should see the newly created runner listed.

🚨 If you do add a runner using the --docker-privileged then you’ll probably want to be selective about what repositories have access to that runner. For example, it would probably be advisable to assign specific repositories to the runner rather than entire groups. Also, importantly, you’ll want to disable any shared or group runners for those repositories so that they end up actually using this privileged runner.

Runner Client

The GitLab Runner client can be used to manage the active runners.

To get a list of available commands use the --help argument.

docker exec -it gitlab-runner gitlab-runner --help

Listing Runners

To list the configured runners, use the list command.

docker exec -it gitlab-runner gitlab-runner list

Typical output might look something like this:

Runtime platform                 arch=amd64 os=linux pid=66 revision=775dd39d version=13.8.0
Listing configured runners                         ConfigFile=/etc/gitlab-runner/config.toml
a9a3dd7b82bd              Executor=docker Token=bxSoZytpPyWZLhB2MQQA URL=https://gitlab.com/
9bad45a61010              Executor=docker Token=AQbMBaySx4qj8236uYFR URL=https://gitlab.com/

Stopping, Starting and Restarting the Runners

You can stop the runners using the stop command.

docker exec -it gitlab-runner gitlab-runner stop

And start them using the start command.

docker exec -it gitlab-runner gitlab-runner start

You can perform both actions in sequence with the restart command.

docker exec -it gitlab-runner gitlab-runner restart

Deregister a Runner

Deregister by name.

docker exec -it gitlab-runner gitlab-runner unregister --name a9a3dd7b82bd

If you have already removed the runner from the GitLab instance then that won’t work and you’ll get an error. However, this can be easily resolved using the following command.

docker exec -it gitlab-runner gitlab-runner verify --delete

Configuration File

The config.toml configuration file will be accessible at /srv/gitlab-runner/config/config.toml on the host machine.

Some global options that you might want to tweak:

  • log_level
  • log_format
  • wait_for_services_timeout — how long to wait for services to become available;
  • concurrent — the total number of jobs which can be run concurrently across all runners (default: 1); and
  • check_interval — how long to wait (in seconds) between checks for new jobs.

Each of the configured runners will have a section in this file which looks something like this:

concurrent = 2
check_interval = 0
connection_max_age = "15m0s"
shutdown_timeout = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "a9a3dd7b82bd"
  url = "https://gitlab.com/"
  token = "bxSoZxtpQyWZLhB2LQGA"
  executor = "docker"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
    [runners.cache.azure]
  [runners.docker]
    tls_verify = false
    image = "alpine"
    privileged = true
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/cache"]
    shm_size = 0

More information on these configuration parameters can be found here. Worth mentioning now:

  • concurrent (global) — The maximum allowed number of simultaneous jobs across all runners. Set this accordingly for the capacity of the machine.

If you want to remove a runner then you can also just delete the corresponding section from config.toml.

With your own runner you’re no longer constrained by the limitations of the (totally awesome!) GitLab shared runners. You can run jobs with impunity, not giving a moment’s thought to running into those resource limits.

Use CI/CD and prosper!