Using IGWN credentials with HTCondor¶
If your workflow jobs require access to restricted IGWN services, you may need to configure your job to include an authorisation credential.
There are two types of credentials you can use with HTCondor jobs.
Kerberos¶
Work in progress
Documenting best-practice for using Kerberos credentials in an HTCondor workflow is a work in progress.
Please consider contributing to help us complete this section.
SciTokens¶
SciTokens are capabiltity tokens that inform services that the bearer of the token should be allowed access to a specific capability (e.g. read) on a specific service. See SciTokens for more details on what SciTokens are and how to use them in general.
SciTokens roll-out is a Work In Progress
The IGWN roll-out of SciTokens is a work in progress, so some details below may be subject to change, especially the values for igwn_oauth_permissions
in the examples below.
This page will be updated as required to reflect the official CompSoft recommendations.
Using a single token in a job¶
To use a SciToken in an HTCondor job, add these command to your HTCondor submit instructions:
use_oauth_services = igwn
igwn_oauth_resource = <service-url>
igwn_oauth_permissions = <capability>
Where
<service-url>
is the fully-qualified URL of the service to access. This is also referred to as the 'audience' (aud
) of the token.<capability>
is the access level that is needed. This is also referred to as the 'scope' (scope
) of the token.
For example, to enable queries to the GWDataFind service located at https://datafind.ligo.org
you would use:
use_oauth_services = igwn
igwn_oauth_resource = https://datafind.ligo.org
igwn_oauth_permissions = gwdatafind.read
With these instructions, HTCondor would automatically generate a new token for you, and would transfer it to the execute node into the directory defined by $_CONDOR_CREDS
. For a single token job like above, the token filename will be igwn.use
:
Single token path
For single-token jobs, the token will be generated on the execute machine as
$_CONDOR_CREDS/igwn.use
You shouldn't need to manually discover the token
Most IGWN scitoken clients should be able to automatically discover the appropriate token file inside the $_CONDOR_CREDS
directory, so you shouldn't actually need to care where the token file exists on the execute node.
Using multiple tokens in a job¶
To use multiple SciTokens in an HTCondor job, modify the above single token example to specify a tag to your igwn_oauth_resource
and igwn_oauth_permisions
commands:
use_oauth_services = igwn
igwn_oauth_resource_token1 = <service-url-1>
igwn_oauth_permissions_token1 = <capability-1>
igwn_oauth_resource_token2 = <service-url-2>
igwn_oauth_permissions_token2 = <capability-2>
For example, to enable queries to GWDataFind at https://datafind.ligo.org
and to GraceDB at https://gracedb.ligo.org
in the same job:
use_oauth_services = igwn
igwn_oauth_resource_gwdatafind = https://datafind.ligo.org
igwn_oauth_permissions_gwdatafind = gwdatafind.read
igwn_oauth_resource_gracedb = https://gracedb.ligo.org
igwn_oauth_permissions_gracedb = gracedb.read
With multiple tokens configured, the name of each token file includes the tag, e.g. igwn_gwdatafind.use
.
Multiple token paths
For the multiple tokens configuration above, the tokens would be generated on the execute machine as
$_CONDOR_CREDS/igwn_gwdatafind.use
$_CONDOR_CREDS/igwn_gracedb.use
X.509¶
X.509 is a credential standard used to encode an identity so that a service can authenticate a request and enable capabilities based on its own records of what users should be allowed to do.
SciTokens are recommended over X.509 in almost all cases
Identity-based X.509 credentials are deprecated in favour of capability-based SciTokens in almost all cases, so please consider using the instructions for tokens above.
For details on which use cases still required X.509 over SciTokens, please contact the Computing and Software Working Group (compsoft@ligo.org).
Using an X.509 credential in a job¶
Generate the X.509 credential manually
Using X.509 with HTCondor requires manually generating the credential before submitting the job.
Please see How to generate a credential for documentation on how to generate an X.509 credential.
To use an X.509 credential file in an HTCondor job, add one of the following commands to your submit instructions:
To automatically discover the credential file based on your environment:
use_x509userproxy = true
To manually specify the path of the credential file:
x509userproxy = /path/to/myproxy.pem
In either case, the credential will be transferred onto the execute machine with your job and its path encoded in the $X509_USER_PROXY
environment variable.