The shared memory device, /dev/shm, provides a temporary file storage filesystem using RAM for storing files. It’s not mandatory to have /dev/shm, although it’s probably desirable since it facilitates inter-process communication (IPC).
Why would you use /dev/shm instead of just stashing a temporary file under /tmp? Well, /dev/shm exists in RAM (so it’s fast), whereas /tmp resides on disk (so it’s relatively slow).
BASH Demo
You can create temporary files in shared memory directly from BASH. First let’s check what’s currently under /dev/shm.
ls -l /dev/shm
total 0
Nothing. Cool, we’ll write the current date and time to a file under /dev/shm.
date >/dev/shm/date.txt
Check if the file is there.
ls -l /dev/shm/
-rw-rw-r-- 1 wookie wookie 29 Nov 8 05:40 date.txt
Indeed it is. The file can then be accessed from another process.
cat /dev/shm/date.txt
Mon 08 Nov 2021 05:40:20 GMT
So the shared memory behaves just like a normal file system, but it’s all in RAM.
What Processes are Using Shared Memory?
Many applications make use of shared memory. We can delve into this using the ipcs command. We’ll start by looking at the shared memory segments.
ipcs -m
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x00000000 1376256 wookie 600 7802880 2 dest
0x00000000 1277954 wookie 600 851968 2 dest
0x00000000 1277956 wookie 600 851968 2 dest
0x00000000 1277958 wookie 600 77824 2 dest
0x00000000 1671189 wookie 600 36864 2 dest
0x00000000 1179651 wookie 600 8011776 2 dest
0x00000000 1703959 wookie 600 524288 2 dest
0x0001eaea 557069 wookie 666 16384 0
There are six shared memory segments of various sizes. They have permissions of either 600 (read and write for user) or 666 (read and write for user, group and other). How do those segments relate to processes?
ipcs -pm
------ Shared Memory Creator/Last-op PIDs --------
shmid owner cpid lpid
1376256 wookie 268958 1923
1277954 wookie 6254 285824
1277956 wookie 6254 285824
1277958 wookie 6254 285824
1671189 wookie 6139 504020
1179651 wookie 24234 285810
1703959 wookie 282127 479099
557069 wookie 125674 125674
The cpid column gives the PID of the process that created the shared memory segment, while the lpid column reflects the PID of the last process which interacted with it.
Those shared memory segments were created by four distinct processes. If we look up the PIDs this is what we find:
268958 /usr/lib/rstudio/bin/rstudio
6254 /usr/lib/firefox/firefox
6139 /opt/google/chrome/chrome
24234 /usr/lib/thunderbird/thunderbird
282127 /usr/lib/slack/slack
125674 /usr/share/dbeaver-ce/dbeaver
Many desktop applications (for example, Slack, Firefox, RStudio, Thunderbird and DBeaver) use shared memory. Some applications (like Firefox) use multiple segments of shared memory.
Now it stands to reason that if these applications are using shared memory on my desktop then they will probably want to use it in a Docker container too. So we need to ensure that there is shared memory available in a container.
What about Docker?
Docker containers are allocated 64 MB of shared memory by default. We’ll fire up an Ubuntu container to test.
docker run --rm -it --name ubuntu ubuntu
Now we can check the characteristics of that container using docker inspect.
docker inspect ubuntu | grep -i shm
"ShmSize": 67108864,
Nice: 64 MB is precisely 67108864 bytes!
Can we change the amount of shared memory allocated to a container? Sure we can! Use the --shm-size option.
docker run --rm -it --name ubuntu --shm-size=2gb ubuntu
Inspect the container again.
docker inspect ubuntu | grep -i shm
"ShmSize": 2147483648,
Aha: 2 GB is exactly 2147483648 bytes.
In both of the cases above the container is getting its own /dev/shm, separate from that of the host. To confirm, let’s check on its contents.
ls -l /dev/shm
total 0
It’s pristine: no sight of the file we created in /dev/shm on the host.
Mounting Host /dev/shm in a Container
What about sharing memory between the host and a container or between containers? This can be done by mounting /dev/shm.
docker run --rm -it --name ubuntu -v /dev/shm:/dev/shm ubuntu
Now we can see the file we created earlier from within the container.
cat /dev/shm/date.txt
Mon 08 Nov 2021 05:40:20 GMT
Shared Memory & Selenium
I make rather extensive use of Selenium Docker images. For anything but simple applications these have a tendency to be rather memory intensive.
Launch Selenium/Firefox and allow access to 2 GB of shared memory.
docker run -d -p 4444:4444 --shm-size=2gb selenium/standalone-firefox:3.141
Launch Selenium/Chrome and volume mount the host’s shared memory.
docker run -d -p 4444:4444 -v /dev/shm:/dev/shm selenium/standalone-chome:3.141
Chrome can be particularly resource hungry, so I often disable its use of shared memory via the --disable-dev-shm-usage option.
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("--disable-dev-shm-usage")
driver = webdriver.Remote(
command_executor="http://127.0.0.1:4444/wd/hub",
desired_capabilities=DesiredCapabilities.CHROME,
options=options,
)
Cleaning Up
Just to keep things neat and tidy, we’ll delete the files that we created in /dev/shm.
rm /dev/shm/date.txt