Skip to content

Faster CI #682

@alimanfoo

Description

@alimanfoo

As part of #366 we have been moving all Anopheles tests to use simulated data, rather than accessing data from GCS. The action which run these new unit tests (tests.yml) are relatively fast, e.g., running in ~6 minutes. They also have the benefit that they don't need to access GCS data and so can be run from a PR from a fork.

The action which runs the remaining legacy tests that access GCS data are defined in legacy_tests.yml. These tests have been gradually removed during the refactoring work, and at some point we might be able to completely remove this action. Until that time, this action currently takes ~13m and should reduce in time with further refactoring ongoing (e.g., via #665).

The slowest action currently is the notebooks.yml action which runs all Jupyter notebooks in the "notebooks" folder. This takes around 41m and accesses data from GCS so cannot be run in a PR from a fork.

It would be great in general if checks that run in PRs run faster and can be run in a PR from a fork.

Here's a suggested approach:

  • Move the notebooks.yml action to only run as part of the release.yml action. I.e., we only run the notebooks when trying to make a release. Any notebook failures will block a release being uploaded to PyPI, and so we can then fix any problems and retag the release.

  • Move the legacy_tests.yml action to only run as part of the release.yml action, for the same reason as above.

  • At a later date, when all of the Anopheles unit tests that access GCS data have been removed via the refactoring work, maybe remove the legacy_tests.yml action completely. Although possibly some Plasmodium tests will remain that still access GCS data. In which case rename legacy_tests.yml to integration_tests.yml to help better clarify the purpose.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions