Skip to content

Containerisation of software

Containers provide a means to install and run your analysis code in a self-contained environment with all required dependencies, somewhat like a virtual machine. Unlike a virtual machine, however, a container uses the host system's underlying operating system, reducing the disk size of the container image and improving performance.

We make use of two containerization platforms:

  • docker: the best-known and most widely integrated platform, requires administrator priveleges to use.
  • singularity: a free, open-source platform which does not require adminstrator priveleges and is more suited to high-performance and high-throughput computing projects. Singularity can run docker images, but not vice-versa.

Typically an analysis pipeline will use continuous integration to build and push a docker image to a container registry (like dockerhub and/or Once an image is in a publicly-accessible registry it can be published to CVMFS, making it available to various high-thoughtput computing resources running singularity.

Creating A Docker Image

Detailed instructions for working with the container registry can be found on the computing wiki.

In short, developers are required to:

  • Host their software in the gitlab repository at
  • Create a recipe, in the form of a Dockerfile, which installs all software dependencies and reproduces the desired runtime environment
  • Build and push the image created by the Dockerfile to
  • Ideally: integrate the image build into the project's CI-pipeline such that appropriately tagged container images maintain parity with the software in the repository automatically.

To make the container image available on high-throughput resources, they should be published to CVMFS.

Publishing Singularity Images To CVMFS

Power Users

To publish a docker image to CVMFS:

  1. Fork this repository
  2. Add the registry URL for your image to docker_images.txt
  3. Submit a pull request

NOTE: the interaction of the github and gitlab APIs apparently does not allow wildcards (*) in image names. You must explicitly state which image tag(s) to publish.

Everyone else

The Open Science Grid (OSG) consortium maintains a service which automatically converts publicly-available docker images to singularity and publishes them to CVMFS. Taking advantage of this service is as simple as a 1-line change to a text file in an OSG github repository, through a standard pull request.

Suppose you have a docker image in the LIGO gitlab registry you would like to access in CVMFS. Let's call that image:

Fork the OSG CVMFS singularity synchronisation repository (click "Fork")

Clone your fork:

git clone

Set upstream and create a branch:

cd cvmfs-singularity-sync
git remote add upstream
git checkout -b my-awesome-image

Update docker_images.txt to include your image. You can see some examples from our community already:

$ grep -A 3 'LIGO - user defined images'  docker_images.txt
# LIGO - user defined images

Add your image around here. Commit and push your new branch to your fork:

$ git add docker_images.txt
$ git commit -m "Added Albert Einstein's awesome image to CVMFS publishing via the OSG"
$ git push origin my-aweseom-image
Counting objects: 5, done.
Delta compression using up to 48 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 368 bytes | 0 bytes/s, done.
Total 3 (delta 2), reused 0 (delta 0)
remote: Resolving deltas: 100% (2/2), completed with 2 local objects.
remote: Create a pull request for 'my-awesome-image' on GitHub by visiting:
 * [new branch]      my-awesome-image -> my-awesome-image

Follow the github URL to open a pull request on the OSG github repository to merge your changes upstream.