Skip to content

IGWN Job Accounting

Accounting tags are required

All jobs submitted to IGWN computing resources are required to provide accounting information, otherwise they will be rejeced instantly.

In order to keep track of how much computing is used for IGWN data processing - both to analyse current usage, and anticipate future needs - all jobs that are submitted must provide an IGWN Accounting Tag that idenfies the analysis being performed.

IGWN Accounting Tags

An IGWN Accounting tag is a period-delimited string following this schema

igwn.<purpose>.<observing run>.<group>.<pipeline>

with the following components:

  • <purpose>: one of
    • dev: development
    • prod: production
    • sim: simulations
  • <observing run>: the lowercase observing run ID, e.g. o3 (Observing Run 3)
  • <group>: the lowercase common short name of the IGWN Working Group for which the work is performed, e.g. calibration
  • <pipeline>: a short period-delimited ID for the pipeline being executed

For example:

igwn.prod.o4.calibration.online.gstlal

Specifying accounting information

All jobs must include the following command when submitting jobs:

accounting_group = <accounting-tag>

For example:

accounting_group = igwn.prod.o4.calibration.online.gstlal

Additionally, if the jobs are submitted from a shared user account, e.g. cbc, then accounting_group_user must be included to indicate which human being is responsible for those jobs, e.g:

accounting_group = igwn.prod.o4.calibration.online.gstlal
accounting_group_user = marie.curie

Selecting or requesting an accounting tag

If you are unsure which accounting tag to use, first ask within your data analysis group. If no one in that group is sure, then you should contact the chair of the working subgroup or group in which your analysis is performed.

To request a new accounting tag, open a helpdesk ticket and select the AccountingTagRequest template from the pull-down menu of templates. Make sure that you fill out all of the information which that template tells you is required. In particular, you will need to provide:

  • The purposes (one or more of prod, dev, or sim)
  • The observing run(s) (e.g. o4, o5; you may request for more than one on a single ticket)
  • The working group (e.g. cw or calibration)
  • The pipeline, period-delimited (e.g. online.gstlal)

In addition, for each tag you request, you must also provide an estimate of the CPU core hours you expect to use. If this estimate is sufficiently large, then you may be required to perform a more careful estimate, and possibly also to benchmark and profile your code. Your entries to this table might look like:

Tag CPU Core hours
igwn.dev.o4.cbc.online.gstlal 150,000
igwn.prod.o4.cbc.online.gstlal 750,000
igwn.dev.o5.cbc.online.gstlal 250,000

Finally, when you open this issue, you should tag all of the chairs of your working group, using the syntax @marie.curie, so that they are aware. The next step will be for those chairs to comment on the ticket and indicate that they approve, and to then tag the chairs of the division, who will indicate their approval and also which priority level to give this analysis. They then tag the computing chairs, who will review the approvals and assign it for deployment.

To see current usage summarized by accounting tag, visit:

https://accounting.igwn.org

Full example

accounting.sub

executable = /usr/bin/od
arguments = -An -N2 -i

input = /dev/random
output = /dev/null

accounting_group = igwn.prod.o4.calibration.online.gstlal

request_disk = 1MB
request_memory = 100MB

queue