# Fixing Truncated Logs on Gitlab CI/CD

I’ve got a few CI/CD jobs running on GitLab which produce long logs. So much so that the logs get truncated.

Job's log exceeded limit of 4194304 bytes.


There’s a fundamental problem with this though: if something’s going to break then it’s inevitably going to happen after the logs have been truncated so I won’t be able to actually see what’s broken.

Fortunately this is easily fixed, provided that you have access to the configuration for your GitLab Runner. At Fathom Data we’ve got a dedicated EC2 instance with GitLab Runner on Docker.

docker ps --format "table {{.ID}}\t{{.Image}}\t{{.Names}}"

CONTAINER ID   IMAGE                                    NAMES
6e14e22024a9   gitlab/gitlab-runner:latest              gitlab-runner


First we’ll need to fire up BASH on that container.

docker exec -it gitlab-runner /bin/bash


Now edit the GitLab Runner configuration in config.toml.

vim /etc/gitlab-runner/config.toml


Find the section for the runner in question and insert an output_limit entry (units are kilobytes, with a default of 4096). I bumped mine up to 16384 (16 MB) and this proved to be sufficient. YMMV.

[[runners]]
name = "Verbose"
url = "https://gitlab.com/"
executor = "docker"
output_limit = 16384
[runners.custom_build_dir]
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.docker]
tls_verify = false
image = "alpine"
privileged = true
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/cache"]
shm_size = 0