Skip to content

HTCondor

The IGWN Computing Grid is based on top of HTCondor, a specialized workload management system for compute-intensive jobs. HTCondor is used to specify discrete work units (jobs) you want completed that are then distributed across the available resources with sophisticated scheduling, prioritisation, monitoring, and reporting capabilities.

Basic HTCondor Usage

For an excellent introduction to HTCondor usage, please refer to the official HTCondor Users' Manual:

https://htcondor.readthedocs.io/en/lts/users-manual/

Advanced users may find the HTCondor manual's "recipes" page particularly useful:

https://htcondor.readthedocs.io/en/latest/faq/index.html

In addition, the Center for High Throughput Computing in Madison maintain a youtube channel with a variety of HTCondor tutorials for both users and administrators:

https://www.youtube.com/channel/UCd1UBXmZIgB4p85t2tu-gLw

For introductory user tutorials to HTCondor, see e.g. the HTCondor User Tutorials playlist:

https://www.youtube.com/watch?v=oMAvxsFJaw4&list=PLO7gMRGDPNumCuo3pCdRk23GDLNKFVjHn

The remainder of this guide describes the non-standard features of the IGWN Computing Grid.

General Distributed Computing Documentation

For additional introductory material on HTCondor we also recommend the excellent OSPool documentation created by the OSG consortium:

https://portal.osg-htc.org/documentation/

OSPool versus IGWN pool

The OSPool is a general purpose distributed HTCondor pool of resources operated by the OSG consortium. The vast majority of IGWN use cases will use the IGWN pool operated by IGWN-affiliated institutions and documented on the pages you are looking at. As such, the OSpool documentation pages make many references to OSPool-specific features and support channels. For support in the IGWN pool your first point of contact should generally be via the issue tracker on our computing helpdesk. Nonetheless, the OSpool documentation offers excellent and up to date tutorials on many of the common features in both pools, such as HTCondor tutorials, data/software access and containerisation.

Access to the IGWN Computing Grid

See Access point for details on how to access the IGWN Computing Grid.

IGWN Computing Grid features

The following pages describe those features specific to the IGWN Computing Grid.

Mandatory extra features

The following extra considerations are MANDATORY:

Optional extra features

The following extra considerations are OPTIONAL, depending on the requirements of your workflow:

Condor-LIGO Mailing list

There is a mailing-list shared between Condor developers and IGWN to discuss IGWN's use of Condor and any problems. To subscribe, go to

https://lists.aei.mpg.de/cgi-bin/mailman/listinfo/condorligo