Software change requests with the SCCB¶
Procedure enhancement proposals
To propose an enhancement to the procedure described below, see Procedure enhancements.
Overview of a request¶
Each request asks for approval to install/update a software package or configuration that can have an impact on users or ongoing analyses. In order that changes are minimally disruptive, the following procedure has been created to ensure that changes are transparent, and that their potential impact has been tested and is understood.
The basic procedure is defined by the status
scoped label set
Request procedure¶
Creating a request¶
Click here to open a new Request
Embrace the request template
Please ensure that you complete the template as fully as you can, to enable efficient handling of your request, including giving a description of the changes and their potential impact on downstream users.
The following labels should be added to all newly opened requests:
The status::validation label
When to apply: as soon as a request is opened.
Who should apply: the SCCB.
What happens next: packaging, review, and testing.
The review::required label
When to apply: as soon as a request is opened.
Who should apply: the SCCB.
What happens next: the review team will be contacted about the status of the review.
Additionally, the necessary packaging request labels will be applied (e.g. rhel::requested)
Expedited requests¶
If you require expedited approval for immediate deployment (e.g. of a critical bug fix), please post a comment to that effect in the comments of the ticket.
The expedite label
When to apply: if this request needs to be approved and deployed immediately.
Who should apply: the requester.
What happens next: the SCCB will try and push a request through as fast as possibly, whilst still following each step of the procedure.
Validation¶
The status::validation stage of a request includes a few parallel tracks:
- Packaging (if appropriate)
- Testing
- Scientific review
Once these steps are all complete, the request moves to be considered formally by the SCCB.
The status::requested label
When to apply: once all review, packaging, and testing are complete.
Who should apply: the requester.
What happens next: the SCCB will vote.
Packaging¶
For each of the options selected in the Distributions section of the request issue, package builds will be coordinated by the Computing and Software group's packaging team.
For each distribution there is a set of labels that track the progress of the packaging for that distribution, e.g:
| Label | Description |
|---|---|
| platform::requested | Requires packaging for this platform |
| platform::packaged | This package is available for this platform |
| platform::testing | This package is installed in the testing environment |
| platform::tested | This package has been tested in the testing environment for this platform |
| platform::staging | This package is ready for inclusion in the production environment |
| platform::production | This package is installed in the production environment |
The platform::stage packaging labels
When to apply: at the relevant stage in the packaging process.
Who should apply: a packaging team member.
The exception to this rule is the tested label (see below).
Testing¶
Once packages are available on the testing system, they should be tested thoroughly.
Automated testing (continuous integration)
Where possible, all software projects should employ automated testing, or continuous integration, to prevent harmful changes entering the codebase as far as possible. Demonstrating that your project has robust automated testing can speed up the testing time by reducing the amount of manual testing required.
The platform::tested labels
When to apply: after a package has been installed in the testing
environment for the relevant platform.
Who should apply: the requester.
For testing, packages will be installed in the following locations:
| Platform | Location |
|---|---|
| Centos 8 | ldas-pcdev16.ligo.caltech.edu |
| Conda | The igwn-pyXY-testing environment(s) |
| Debian | NONE |
| RHEL7 | ldas-pcdev4.ligo.caltech.edu |
Alternative distributions (including AWS, virtualenv, etc.) should be installed into a canary testing environment as soon as packages are available for the requested software.
Don't test in production
Production systems should not be used for testing under any circumstances.
Scientific review¶
All scientific software packages (e.g. those used in an analysis pipeline) must be approved by the relevant scientific review team before the SCCB will consider them for approval.
The review::approved label
When to apply: when the review team approves the requested change
Who should apply: a member of the review team.
Approval¶
SCCB¶
Once the status::requested label has been applied to a ticket, the SCCB membership will vote on whether to approve, or reject, the request.
For details of the SCCB group approval rules, please see ยง4.2 (Approval rules) of the SCCB Charge document LIGO-T1800406-v2.
The status::approved label
When to apply: when the SCCB approves a request.
Who should apply: the SCCB member providing the final approval.
What happens next: the run coordinator will consider the request.
Any member of the SCCB may apply the status::hold label to prevent automatic approval of the request (once it otherwise has enough votes). This should be applied by an SCCB member to address any issues with the request that were not identified during validation.
Run Coordinator¶
Once approved, if the request was made during an observing epoch, the request will be considered by the Run Coordinator(s). They may also post a comment on the ticket to indicate a preference as to how, or when, the new software should be deployed.
The status::deploy label
When to apply: when the Run Coordinator approves a request.
Who should apply: the lead Run Coordinator.
What happens next: the change will be deployed.
Deployment¶
When the status::deploy label is applied, the requested change can be deployed.
Unless special action is required, the following is the timeline for deployment of approved software, regardless of the deployment mechanism:
Scheduled deployment
If the request is approved by 12:00 Pacific Time (Los Angeles) on a Friday, it will be considered for deployment during the next available Tuesday maintenance slot.
Otherwise it will not be deployed and will be considered for a future maintenance period.
Exceptions can be discussed on the Monday JRPC Run-Coordination meeting.
Managed deployments will be coordinated with the CompSoft packaging team with the following labels:
- conda::production
- debian::production
- rhel::production
Exceptional deployment
expedite requests will be considered for exceptional deployment at the next available opportunity. These will be scheduled on a case-by-case basis by the Run Coordinator.
The status::deployed label
When to apply: when the requested software has been deployed on the target production system
Who should apply: the individual who undertakes the deployment.
What happens next: the SCCB will wait for feedback before closing the ticket.
Closing a ticket¶
Tickets will be closed once the relevant software/configuration has been deployed onto the production system, or the request has been rejected.
After the status::deployed label has been applied, a nominal waiting period will be observed, allowing users to report issues with the newly deployed software, before the issue is closed.
Closing tickets
When to close: when production deployments have been deemed successfull.
Who should close: the SCCB
Useful information¶
Contact¶
If you need to contact any the SCCB directly about your request, either use the
@sccb gitlab syntax in your message, or email sccb@ligo.org.
Labels¶
For a full overview of the many labels that may be applied to a request, see https://git.ligo.org/computing/sccb/-/labels?sort=name_asc.