Skip to content

python/test

Configures jobs to test a Python project.

Description

This component installs the target Python project and then runs pytest to execute the project test suite.

Usage

include:
  - component: git.ligo.org/computing/gitlab/components/python/test@<VERSION>

Inputs

Input Default value Description
job_name sdist The name to give the job.
stage test Pipeline stage to add jobs to
python "python" Name/path of Python interpreter to use
needs [] List of jobs whose artifacts are needed by the test jobs
install_target "." Path/file/package to install (supports wildcards)
install_extra None Name of the extra feature group(s) to install (comma-separated)
pytest_options "" Extra options to pass to pytest
extra_test_commands [] Extra test commands to run (after pytest)
coverage_rcfile None Path to coverage.py configuration file (helps with providing accurate coverage measurements, autodiscovered in most cases)

Notes

Automatic test and coverage reporting

The tests are executed using pytest and automatically include coverage reporting using pytest-cov and Unit test reports.

Coverage is combined in the .post pipeline stage

GitLab's default coverage parser averages coverage values for multiple jobs. This down-ranks coverage from projects that may have lower coverage on some platforms (e.g. Windows).

To get around this limitation, the python/test component configures a python_coverage job in the .post that uses coverage.py's combine command to correctly merge coverage reports from multiple jobs. This job then presents a single coverage statistic for the whole pipeline.

Customisation

pytest

pytest's behaviour can be be customised in one of the following ways (ordered by preference, most recommended to least):

  • by specifying the tool.pytest.ini_options table in the project pyproject.toml configuration file.

    Configure pytest in pyproject.toml

    [tool.pytest.ini_options]
    addopts = "-ra --cov myproject"
    
  • give the pytest_options input

    Configure pytest through the component inputs

    include:
      - component: git.ligo.org/computing/gitlab/components/python/test@<VERSION>
        inputs:
          pytest_options: "-ra -v"
    
  • set the PYTEST_ADDOPTS variable for the python_test job:

    Set PYTEST_ADDOPTS for the python_test job

    include:
      - component: git.ligo.org/computing/gitlab/components/python/test@<VERSION>
    
    python_test:
      variables:
        PYTEST_ADDOPTS: "-k 'not bad_test'
    

coverage.py

Coverage gathering and reporting should be customised via the [tool.coverage] section of the project pyproject.toml configuration file.

Configure coverage.py in pyproject.toml

[tool.coverage.paths]
# map paths from installed locations back to project source
source = [
  "myproject/",  # <-- source path
  "*/myproject/",  # <-- any installed path
]

[tool.coverage.report]
precision = 1
# omit auto-generated version file
omit = [
  "*/_version.py",
]

Examples

Testing a Python project

Run tests for a Python project

include:
  - component: git.ligo.org/computing/gitlab/components/python/test@<VERSION>
    inputs:
      install_extra: "test"
      pytest_options: "-ra -v"
      python_versions:
        - "3.10"
        - "3.11"
        - "3.12"

Executing tests from an installed library

To run tests for an installed library, rather than from the project directory, specify install_target to install the distribution and pytest_options to target it using the --pyargs option:

Running tests for an installed library

include:
  - component: git.ligo.org/computing/gitlab/components/python/wheel@<VERSION>
    inputs:
      project_dir: example/
  - component: git.ligo.org/computing/gitlab/components/python/test@<VERSION>
    inputs:
      job_name: "pytest"
      install_target: "*.whl"
      install_extra: "test"
      needs: [wheel]
      pytest_options: "-ra --pyargs example_project"
      extra_test_commands:
        - my_cli --help
      python_versions:
        - "3.10"
        - "3.11"
        - "3.12"