Skip to content

Look into caching .tox in GitHub actions #2

Closed

Description

@jonathansick

If we can cache .tox in GitHub Actions we could speed up the CI a lot. However, in practice, I find that Tox won't run from a cached .tox directory. It could be worth looking into this.

The workflow is:

name: Python CI

on: [push]

jobs:
  build:

    runs-on: ubuntu-latest
    strategy:
      matrix:
        python:
          - 3.7
          - 3.8

    steps:
      - uses: actions/checkout@v2

      - name: Set up Python
        uses: actions/setup-python@v1
        with:
          python-version: ${{ matrix.python }}

      - name: Install tox
        run: pip install tox

      - name: Cache tox environments
        id: cache-tox
        uses: actions/cache@v1
        with:
          path: .tox
          # setup.cfg, pyproject.toml, and .pre-commit-config.yaml have
          # versioning info that would impact the tox environment. hashFiles
          # only takes a single file path or pattern at the moment.
          key: ${{ runner.os }}-${{ matrix.python }}-tox-${{ hashFiles('setup.cfg') }}-${{ hashFiles('pyproject.toml') }}-${{ hashFiles('.pre-commit-config.yaml') }}

      - name: Run tox
        run: tox -e py,lint,typing  # run tox using Python in path

The error is:

 Run tox 1s
15
##[error]Process completed with exit code 2.
1
Run tox -e py,lint,typing
6
ERROR: invocation failed (exit code 1), logfile: /home/runner/work/safir/safir/.tox/.package/log/.package-4.log
7
================================== log start ===================================
8
Traceback (most recent call last):
9
  File "/opt/hostedtoolcache/Python/3.8.1/x64/lib/python3.8/site-packages/tox/helper/build_requires.py", line 7, in <module>
10
    backend = __import__(backend_spec, fromlist=[None])
11
ModuleNotFoundError: No module named 'setuptools'
12

13
=================================== log end ====================================
14
ERROR: FAIL could not package project - v = InvocationError("/home/runner/work/safir/safir/.tox/.package/bin/python /opt/hostedtoolcache/Python/3.8.1/x64/lib/python3.8/site-packages/tox/helper/build_requires.py setuptools.build_meta ''", 1)
15
##[error]Process completed with exit code 2.

Activity

  1. rra commented on Mar 16, 2022

    @rra
    Member

    We are now caching the tox environments and everything seems to work fine, so presumably this is a bug that was subsequently fixed.

  2. hughesadam87 commented on Sep 27, 2023

    @hughesadam87

    @rra you're caching tox like this or some other way?

      - name: Cache tox environments
        id: cache-tox
        uses: actions/cache@v1
        with:
          path: .tox
    
  3. rra commented on Sep 27, 2023

    @rra
    Member

    We're using the https://github.com/lsst-sqre/run-tox shared action now, but internally it does the following:

        - name: Cache tox environments
          id: cache-tox
          uses: actions/cache@v3
          if: fromJSON(${{ inputs.use-cache }})
          with:
            path: ${{ inputs.working-directory }}/.tox
            # setup.cfg and pyproject.toml have versioning info that would
            # impact the tox environment.
            key: ${{ inputs.cache-key-prefix}}-${{ inputs.python-version }}-${{ hashFiles('pyproject.toml', 'setup.cfg') }}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions