Skip to content

Containerisation of software

Containers provide a means to install and run your analysis code in a self-contained environment with all required dependencies, somewhat like a virtual machine. Unlike a virtual machine, however, a container uses the host system's underlying operating system, reducing the disk size of the container image and improving performance.

We make use of two containerization platforms:

  • docker: the best-known and most widely integrated platform, requires administrator priveleges to use.
  • singularity: a free, open-source platform which does not require adminstrator priveleges and is more suited to high-performance and high-throughput computing projects. Singularity can run docker images, but not vice-versa.

Typically an analysis pipeline will use continuous integration to build and push a docker image to a container registry (like dockerhub and/or containers.ligo.org). Once an image is in a publicly-accessible registry it can be published to CVMFS, making it available to various high-thoughtput computing resources running singularity.

Creating A Docker Image

Detailed instructions for working with the containers.ligo.org container registry can be found on the computing wiki.

In short, developers are required to:

  • Host their software in the gitlab repository at git.ligo.org
  • Create a recipe, in the form of a Dockerfile, which installs all software dependencies and reproduces the desired runtime environment
  • Build and push the image created by the Dockerfile to containers.ligo.org
  • Ideally: integrate the image build into the project's CI-pipeline such that appropriately tagged container images maintain parity with the software in the repository automatically.

To make the container image available on high-throughput resources, they should be published to CVMFS.

Publishing Singularity Images To CVMFS

Power Users

To publish a docker image to CVMFS:

  1. Fork this repository
  2. Add the registry URL for your image to docker_images.txt
  3. Submit a pull request

NOTE: the interaction of the github and gitlab APIs apparently does not allow wildcards (*) in image names. You must explicitly state which image tag(s) to publish.

Everyone else

The Open Science Grid (OSG) consortium maintains a service which automatically converts publicly-available docker images to singularity and publishes them to CVMFS. Taking advantage of this service is as simple as a 1-line change to a text file in an OSG github repository, through a standard pull request.

Suppose you have a docker image in the LIGO gitlab registry you would like to access in CVMFS. Let's call that image:

containers.ligo.org/albert.einstein/my-awesome-image:sometag

Fork the OSG CVMFS singularity synchronisation repository (click "Fork")

Clone your fork:

git clone git@github.com:astroclark/cvmfs-singularity-sync.git

Set upstream and create a branch:

cd cvmfs-singularity-sync
git remote add upstream git@github.com:opensciencegrid/cvmfs-singularity-sync.git
git checkout -b my-awesome-image

Update docker_images.txt to include your image. You can see some examples from our community already:

$ grep -A 3 'LIGO - user defined images'  docker_images.txt
# LIGO - user defined images
containers.ligo.org/joshua.willis/pycbc:latest
containers.ligo.org/james-clark/gwrucio:latest
containers.ligo.org/albert.einstein/my-awesome-image:sometag

Add your image around here. Commit and push your new branch to your fork:

$ git add docker_images.txt
$ git commit -m "Added Albert Einstein's awesome image to CVMFS publishing via the OSG"
$ git push origin my-aweseom-image
Counting objects: 5, done.
Delta compression using up to 48 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 368 bytes | 0 bytes/s, done.
Total 3 (delta 2), reused 0 (delta 0)
remote: Resolving deltas: 100% (2/2), completed with 2 local objects.
remote:
remote: Create a pull request for 'my-awesome-image' on GitHub by visiting:
remote:      https://github.com/albert.einstein/cvmfs-singularity-sync/pull/new/my-awesome-image
remote:
To git@github.com:albert.einstein/cvmfs-singularity-sync.git
 * [new branch]      my-awesome-image -> my-awesome-image

Follow the github URL to open a pull request on the OSG github repository to merge your changes upstream.