fix data cache #555

d-chambers · 2025-09-25T06:15:17Z

Description

This PR tries to fix the data cache in the CI. It uses the data registry to invalidate the cache.

Checklist

I have (if applicable):

referenced the GitHub issue this PR closes.
documented the new feature with docstrings or appropriate doc page.
included a test. See testing guidelines.
your name has been added to the contributors page (docs/contributors.md).
added the "ready_for_review" tag once the PR is ready to be reviewed.

Summary by CodeRabbit

Chores
- Optimized build pipelines with improved caching to speed up environment setup and reduce CI run times.
- Added cache validation and logging to improve reliability and visibility of CI caching behavior.
Tests
- Enabled caching for test, coverage, and docs-build workflows to accelerate test execution and feedback cycles.
- Streamlined dependency installation in CI to make test runs more consistent across workflows.

coderabbitai · 2025-09-25T06:15:24Z

Walkthrough

Introduces caching keyed by a data registry hash in the reusable mamba-install-dascore action, with cache verification and logging. Updates multiple workflows to pass a cache-number input to this action. No public API changes.

Changes

Cohort / File(s)	Summary of Changes
Action: mamba-install-dascore `.github/actions/mamba-install-dascore/action.yml`	Compute DATA_REGISTRY_HASH from `dascore/data_registry.txt`; switch cache step to `actions/cache@v4` with key `data-${{ env.DATA_REGISTRY_HASH }}-${{ inputs.cache-number }}` and restore-keys; add cache status verification and directory listing; reorder printing before final package info.
Workflows consuming action `.github/workflows/get_coverage.yml`, `.github/workflows/run_min_dep_tests.yml`, `.github/workflows/runtests.yml`, `.github/workflows/test_doc_build.yml`	Pass new `cache-number` input to the `./.github/actions/mamba-install-dascore` step (fixed value `1` or from `env.CACHE_NUMBER`).

Possibly related PRs

Ci test docs #504: Modifies the same workflows to add the cache-number input for the mamba-install-dascore action, overlapping with this PR’s changes.

Suggested labels

CI

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title Check	✅ Passed	The title “fix data cache” directly relates to the CI data caching issue addressed in this PR by invalidating the cache via the data registry hash, so it suitably summarizes the main change in a concise manner.
Description Check	✅ Passed	The pull request description follows the repository template by providing a clear Description section explaining the cache fix approach and a matching Checklist section that includes all required items, so it meets the expected structure and content.
Docstring Coverage	✅ Passed	No functions found in the changes. Docstring coverage check skipped.

✨ Finishing touches

🧪 Generate unit tests

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch ci_cache

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

codecov · 2025-09-25T06:19:24Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.92%. Comparing base (018d807) to head (0c4c350).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #555   +/-   ##
=======================================
  Coverage   99.92%   99.92%           
=======================================
  Files         126      126           
  Lines       10472    10472           
=======================================
  Hits        10464    10464           
  Misses          8        8

Flag	Coverage Δ
unittests	`99.92% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

coderabbitai

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 018d807 and 0c4c350.

📒 Files selected for processing (5)

.github/actions/mamba-install-dascore/action.yml (2 hunks)
.github/workflows/get_coverage.yml (1 hunks)
.github/workflows/run_min_dep_tests.yml (1 hunks)
.github/workflows/runtests.yml (1 hunks)
.github/workflows/test_doc_build.yml (1 hunks)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)

GitHub Check: test_code_min_deps (windows-latest, 3.13)
GitHub Check: test_code (ubuntu-latest, 3.11)
GitHub Check: test_code (macos-latest, 3.13)
GitHub Check: Run benchmarks

🔇 Additional comments (4)

.github/workflows/test_doc_build.yml (1)

21-25: LGTM – wiring the cache number input

Thanks for threading the new cache-number into the docs workflow; this keeps the caching behavior consistent with the updated action.

.github/workflows/get_coverage.yml (1)

19-23: LGTM – cache input propagation looks good

Passing the cache number through here keeps coverage runs aligned with the shared caching scheme.

.github/workflows/runtests.yml (1)

51-55: LGTM – using the shared environment knob

Hooking the action up to env.CACHE_NUMBER gives us a single toggle for cache busting across the matrix. Nice touch.

.github/workflows/run_min_dep_tests.yml (1)

46-52: LGTM – cache-number plumbed through

The min-deps workflow now opts into the same cache controls as the others. All good here.

coderabbitai · 2025-09-26T16:15:52Z

.github/actions/mamba-install-dascore/action.yml

+    - name: get data registry hash
+      shell: bash
+      run: |
+        echo "DATA_REGISTRY_HASH=$(sha256sum dascore/data_registry.txt | cut -d' ' -f1)" >> $GITHUB_ENV
+


⚠️ Potential issue | 🔴 Critical

Replace sha256sum with a cross-platform hash computation

The new cache key step runs on macOS runners, where sha256sum is not installed by default. This will cause every macOS job to fail before we even get to caching. Please switch to a portable approach (e.g., Python’s hashlib) so the hash generation works across Linux, macOS, and Windows.

Apply this diff to fix the issue:

- name: get data registry hash - shell: bash - run: | - echo "DATA_REGISTRY_HASH=$(sha256sum dascore/data_registry.txt | cut -d' ' -f1)" >> $GITHUB_ENV + shell: bash + run: | + python - <<'PY' +from pathlib import Path +import hashlib +import os + +data = Path("dascore/data_registry.txt").read_bytes() +hash_value = hashlib.sha256(data).hexdigest() +with open(os.environ["GITHUB_ENV"], "a", encoding="utf-8") as env_file: + env_file.write(f"DATA_REGISTRY_HASH={hash_value}\n") +PY

🤖 Prompt for AI Agents

In .github/actions/mamba-install-dascore/action.yml around lines 67 to 71 the step uses the non-portable `sha256sum` which is missing on macOS runners; replace that shell command with a cross-platform Python invocation that reads dascore/data_registry.txt in binary, computes hashlib.sha256(...).hexdigest(), and appends the result to $GITHUB_ENV (e.g., run a one-liner python -c that prints the hex digest and echo it into DATA_REGISTRY_HASH) so the hash generation works on Linux, macOS, and Windows runners.

codspeed-hq · 2025-09-26T16:40:49Z

CodSpeed Performance Report

Merging #555 will not alter performance

_{Comparing ci_cache (0c4c350) with master (018d807)}

Summary

✅ 49 untouched

fix data cache

6d849c1

Merge branch 'master' of github.com:dasdae/dascore into ci_cache

0c4c350

coderabbitai bot added the CI continuous integration label Sep 26, 2025

coderabbitai bot reviewed Sep 26, 2025

View reviewed changes

d-chambers merged commit 57bb766 into master Sep 26, 2025
25 checks passed

d-chambers deleted the ci_cache branch October 2, 2025 12:44

coderabbitai bot mentioned this pull request Dec 22, 2025

CI: test cache, bump tested version, global config #591

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix data cache #555

fix data cache #555

Uh oh!

d-chambers commented Sep 25, 2025 •

edited by coderabbitai bot

Loading

Uh oh!

coderabbitai bot commented Sep 25, 2025 •

edited

Loading

Uh oh!

codecov bot commented Sep 25, 2025 •

edited

Loading

Uh oh!

coderabbitai bot left a comment

Uh oh!

coderabbitai bot Sep 26, 2025

Uh oh!

codspeed-hq bot commented Sep 26, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

fix data cache #555

fix data cache #555

Uh oh!

Conversation

d-chambers commented Sep 25, 2025 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Summary by CodeRabbit

Uh oh!

coderabbitai bot commented Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Possibly related PRs

Suggested labels

Pre-merge checks and finishing touches

Uh oh!

codecov bot commented Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

coderabbitai bot Sep 26, 2025

Choose a reason for hiding this comment

Uh oh!

codspeed-hq bot commented Sep 26, 2025

CodSpeed Performance Report

Merging #555 will not alter performance

Summary

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

d-chambers commented Sep 25, 2025 •

edited by coderabbitai bot

Loading

coderabbitai bot commented Sep 25, 2025 •

edited

Loading

codecov bot commented Sep 25, 2025 •

edited

Loading