Change control for IGWN Conda¶

This document describes how the contents of the IGWN Conda Distribution is controlled, and how to suggest or request a change.

Environment defintions¶

The contents of each environment is strictly controlled using a hierarchical set of package definitions files. These packages are separated into two groups:

Upstream¶

The upstream packages, being those not directly developed or maintained by the GW community, are controlled in a single file upstream.yaml. The file looks something like the following, with each package specified as name=version:

Upstream packages definition

dependencies:
  - astroplan=0.5
  - astropy=3.2.3
  - autoconf=2.69
  - automake=1.16.1
  - bc=1.07.1
  - beautifulsoup4=4.8.1
  - boost-cpp=1.70.0
  - celery=4.3.0
  - celery-eternal=0.1.1

Upstream package pinning

upstream packages are pinned to a specific version, but not a specific build, the assumption being that a rebuild of an upstream package is likely beneficial. Re-builds that are incompatible with the rest of the IGWN Conda packages would simply not be selected by the conda solver anyway.

IGWN packages¶

The IGWN packages, being those directly developed or maintained by the GW community, are each declared in their own <package>.yaml file. These files follow a similar syntax to a conda build recipe, including the use of selectors. For example, the definition file for ligo.skymap looks like this:

{!packages/ligo.skymap.yaml!}

IGWN package pinning

IGWN packages are pinned to a specific build on all platforms.

Selectors¶

In order to define YAML variables specific to a platform, or python version, pre-processing selectors can be used, see Preprocessing selectors for more details on how these are used by conda-build. The IGWN Conda build tools recognise the following subset of selectors:

py, the <major><minor> python version without dots, e.g. 37
linux
osx

Additionally, {{ py }} can be used as a Jinja2-style substitution variable.

Changing the package list¶

1. Open an SCCB ticket¶

All changes to packages should be preceded by an SCCB request ticket that details the desired change, see Software change requests with the SCCB for full details.

2. Propose changes to the package definitions files¶

Open a merge request that adds/removes/updates the necessary package definitions files.

If you need help, just ask

If you don't feel confident proposing a specific change the definitions files for whatever reason, please post a comment on your SCCB ticket and someone from the IGWN computing team will be happy to propose the change on your behalf.

The changes will be automatically tested, then merged onto the testing git branch, and deployed into the igwn-pyXY-testing conda environments.

Once the SCCB has approved the request, the changes will then be merged onto the staging git branch and deployed into the igwn-pyXY-staging conda environments. A deployment will be scheduled either for the next appropriate opportunity, in the case of an expedited request, or for the next regular maintenance slot.

The deployment will then be finalised by the creation of a new dated git tag, and the a set of stable conda environments will be deployed.