If you have multiple applications accessing OSRM data then it does not make sense for each of those to have a separate copy of the data resident in memory. This is especially true if you’re using a relatively large map, in which case memory consumed by multiple processes might be enormous.
An alternative is to store the map data in shared memory, allowing multiple processes to access a single copy of the data. The official OSRM documentation for using shared memory can be found here. This post gives further details.
Adjust Limits
You need to configure your system first. There are both system-wide and user-specific changes.
System
Check on the default shared memory limits.
$ ipcs -lm
------ Shared Memory Limits --------
max number of segments = 4096
max seg size (kbytes) = 18014398509465599
max total shared memory (kbytes) = 18014398509481980
min seg size (bytes) = 1
Those numbers are pretty enormous and will probably be sufficient. But if you are using a really big map then you might need to expand them. You can do this by setting the shmall and shmmax kernel parameters. We’ll go the other way and actually reduce the allocations. Add something like the following to the end of /etc/sysctl.conf:
kernel.shmall = 1000000000
kernel.shmmax = 10240000000
The shmmax parameter specifies the maximum amount of shared memory (in kB) that can be requested by a single process. The shmall parameter sets the total amount of shared memory pages that can be used across the entire system. A page is simply a block of memory. You can check on the page size (in bytes) as follows:
$ getconf PAGE_SIZE
4096
Now reload the system settings.
sudo sysctl -p
Use ipcs again to check that the changes have been applied.
$ ipcs -lm
------ Shared Memory Limits --------
max number of segments = 4096
max seg size (kbytes) = 10000000
max total shared memory (kbytes) = 4000000000
min seg size (bytes) = 1
User
There are separate limits which apply to individual users.
$ ulimit -a | grep max
max locked memory (kbytes, -l) 16384
max memory size (kbytes, -m) unlimited
max user processes (-u) 7729
To share a map of any reasonable size we’d need to boost those. Add the following to /etc/security/limits.conf (the settings will be applied to the ubuntu user).
ubuntu hard memlock unlimited
# Memory is in units of kB.
ubuntu soft memlock 8388608
Now logout and login again so that the changes take effect. Check.
$ ulimit -a | grep max
max locked memory (kbytes, -l) 8388608
max memory size (kbytes, -m) unlimited
max user processes (-u) 7729
Using Shared Memory
Now we are ready to add OSRM data to shared memory. Suppose that we have data which has already been prepared for use with OSRM.
$ osrm-datastore durban.xml.osrm
Check.
$ osrm-datastore --list
[info] name shm key timestamp size
[info] /static 0 1 16519676
[info] /updatable 1 1 34074239
Now you can launch the OSRM router using data from shared memory.
$ osrm-routed --shared-memory
It’s also possible to have multiple datasets in memory simultaneously. First let’s clean up (this will remove the existing OSRM data in shared memory).
$ osrm-datastore --remove-locks
$ osrm-datastore --spring-clean
Now add two maps, giving each a specific name.
$ osrm-datastore --dataset-name durban durban.xml.osrm
$ osrm-datastore --dataset-name pietermaritzburg pietermaritzburg.xml.osrm
Check.
$ osrm-datastore --list
[info] name shm key timestamp size
[info] durban/static 0 1 16519676
[info] durban/updatable 1 1 34074239
[info] pietermaritzburg/static 2 1 5691727
[info] pietermaritzburg/updatable 3 1 9880003
You can now launch the OSRM router using either of these shared datasets.
osrm-routed --shared-memory --dataset-name=pietermaritzburg
Notes
- According to this thread it’s only possible for OSRM to create shared memory objects for one user. For instance, you cannot run
osrm-datastorefor one map asrootand then again for another map as another user. This will result in a “Permission denied” error. - If you get a “Permission denied” error when running
osrm-datastoreit might help to reboot the machine.