I’ve got a project which takes a long time to build. I’ve been using the shared runners on GitLab. However, the total time constraint has become a serious limitation (ideally I want to rebuild my project fairly regularly).
I’m going to install GitLab Runner as a Docker service on an underutilised EC2 instance.
Background information on Continuous Integration with GitLab can be found here.
Start Runner Container
First create an instance of the GitLab Runner container.
docker run -d --name gitlab-runner --restart always \ -v /srv/gitlab-runner/config:/etc/gitlab-runner \ -v /var/run/docker.sock:/var/run/docker.sock \ gitlab/gitlab-runner:latest
More information about the GitLab Runner can be found here.
Obtain a Token
Either for a single project or a group:
- go to Settings > CI/CD
- expand the Runners section
- copy the token from the Set up a specific Runner manually section.
Register a Runner
To register a runner, use the
docker exec -it gitlab-runner gitlab-runner register
💡 This seems repetitive, but there’s a reason for the repetition: the first
gitlab-runner refers to the running container, while the second reference specifies a command to run within the container.
📌 If you need to use Docker-in-Docker (for example, to build a Docker image as part of the CI/CD process) then you should also specify the
docker exec -it gitlab-runner gitlab-runner register --docker-privileged
Provide the following input:
https://gitlab.com/as the coordinator URL.
- Provide the token.
- Provide a description for the runner.
- Supply tags for the runner (can leave this blank for the moment).
dockeras the executor.
- Specify a default Docker image (for example,
rocker/verse:latest) which will be used if an image is not given in
The runner should be automatically restarted. However, you can restart manually just to be sure:
docker restart gitlab-runner
If you refresh the Settings > CI/CD page you should see the newly created runner listed.
🚨 If you do add a runner using the
--docker-privileged then you’ll probably want to be selective about what repositories have access to that runner. So, for example, it would probably be advisable to assign specific repositories to the runner rather than entire groups. Also, importantly, you’ll want to disable any shared or group runners for those repositories so that they end up actually using this privileged runner.
To list the configured runners, use the
docker exec -it gitlab-runner gitlab-runner list
Typical output might look something like this:
Runtime platform arch=amd64 os=linux pid=66 revision=775dd39d version=13.8.0 Listing configured runners ConfigFile=/etc/gitlab-runner/config.toml a9a3dd7b82bd Executor=docker Token=bxSoZytpPyWZLhB2MQQA URL=https://gitlab.com/ 9bad45a61010 Executor=docker Token=AQbMBaySx4qj8236uYFR URL=https://gitlab.com/
Stopping, Starting and Restarting the Runners
You can stop the runners using the
docker exec -it gitlab-runner gitlab-runner stop
And start them using the
docker exec -it gitlab-runner gitlab-runner start
You can perform both actions in sequence with the
docker exec -it gitlab-runner gitlab-runner restart
Deregister a Runner
Deregister by name.
docker exec -it gitlab-runner gitlab-runner unregister --name a9a3dd7b82bd
config.toml configuration file will be accessible at
/srv/gitlab-runner/config/config.toml on the host machine.
Some global options that you might want to tweak:
concurrent— the number of jobs which can be run concurrently (default: 1); and
check_interval— how long to wait (in seconds) between checks for new jobs.
Each of the configured runners will have a section in this file which looks something like this:
[[runners]] name = "a9a3dd7b82bd" url = "https://gitlab.com/" token = "bxSoZxtpQyWZLhB2LQGA" executor = "docker" [runners.custom_build_dir] [runners.cache] [runners.cache.s3] [runners.cache.gcs] [runners.cache.azure] [runners.docker] tls_verify = false image = "alpine" privileged = true disable_entrypoint_overwrite = false oom_kill_disable = false disable_cache = false volumes = ["/cache"] shm_size = 0
More information on these configuration parameters can be found here.
With your own runner you’re no longer constrained by the limitations of the (totally awesome!) GitLab shared runners. You can run jobs with impunity, not giving a moment’s thought to running into those resource limits.
Use CI/CD and prosper!