fix: source_path not mounted#3738
Conversation
Fixes snakemake source-cache not being bind mounted to container when using additional apptainer-args.
📝 WalkthroughWalkthroughAlways compute the Snakemake source-cache path and, if it exists, add it as a quoted bind mount to Apptainer/Singularity invocations; add a test Snakefile and a test function exercising source-cache mounting with apptainer bind args. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant User as User
participant SM as Snakemake
participant Cont as Apptainer/Singularity
User->>SM: Run workflow (rule uses workflow.source_path)
SM->>SM: Compute source_cache_path
alt source_cache_path exists
SM->>SM: Append quoted --bind '<source_cache_path>' to args
else
SM->>SM: Log debug: source-cache not found
end
SM->>Cont: Launch container with assembled args (including any user apptainer-args)
Cont-->>SM: Container runs and returns result
SM-->>User: Rule completes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
Pre-merge checks and finishing touches❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✨ Finishing touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🧪 Early access (Sonnet 4.5): enabledWe are currently testing the Sonnet 4.5 model, which is expected to improve code review quality. However, this model may lead to increased noise levels in the review comments. Please disable the early access features if the noise level causes any inconvenience. Note:
Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
tests/test_singularity_source_cache/Snakefile (1)
1-9: Minor: name the param for clarity and future-proof formattingExplicitly naming the param avoids relying on the implicit {params} expansion and reads clearer.
rule a: output: "test.out" params: - workflow.source_path("test.txt") + src=workflow.source_path("test.txt") singularity: "docker://bash:5.3.3" shell: - "cp {params} {output}" + "cp {params.src} {output}"
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
tests/test_singularity_source_cache/expected-results/test.outis excluded by!**/*.out
📒 Files selected for processing (3)
src/snakemake/deployment/singularity.py(1 hunks)tests/test_singularity_source_cache/Snakefile(1 hunks)tests/tests.py(1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.py
⚙️ CodeRabbit configuration file
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
Files:
tests/tests.pysrc/snakemake/deployment/singularity.py
🧬 Code graph analysis (2)
tests/tests.py (1)
tests/common.py (2)
run(152-503)dpath(33-36)
src/snakemake/deployment/singularity.py (2)
src/snakemake/workflow.py (1)
source_cache_path(280-290)src/snakemake/common/__init__.py (1)
get_appdirs(118-124)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (55)
- GitHub Check: tests (10, ubuntu-latest, py311)
- GitHub Check: tests (9, ubuntu-latest, py313)
- GitHub Check: tests (10, ubuntu-latest, py313)
- GitHub Check: tests (9, ubuntu-latest, py312)
- GitHub Check: tests (10, windows-2022, py312)
- GitHub Check: tests (10, windows-2022, py313)
- GitHub Check: tests (9, windows-2022, py313)
- GitHub Check: tests (10, ubuntu-latest, py312)
- GitHub Check: tests (8, windows-2022, py313)
- GitHub Check: tests (9, ubuntu-latest, py311)
- GitHub Check: tests (8, windows-2022, py312)
- GitHub Check: tests (8, ubuntu-latest, py311)
- GitHub Check: tests (7, windows-2022, py313)
- GitHub Check: tests (6, ubuntu-latest, py312)
- GitHub Check: tests (5, windows-2022, py312)
- GitHub Check: tests (7, ubuntu-latest, py313)
- GitHub Check: tests (5, windows-2022, py313)
- GitHub Check: tests (9, windows-2022, py312)
- GitHub Check: tests (8, ubuntu-latest, py312)
- GitHub Check: tests (8, ubuntu-latest, py313)
- GitHub Check: tests (7, windows-2022, py312)
- GitHub Check: tests (4, windows-2022, py312)
- GitHub Check: tests (6, windows-2022, py312)
- GitHub Check: tests (7, ubuntu-latest, py312)
- GitHub Check: tests (6, ubuntu-latest, py313)
- GitHub Check: tests (6, windows-2022, py313)
- GitHub Check: tests (6, ubuntu-latest, py311)
- GitHub Check: tests (5, ubuntu-latest, py313)
- GitHub Check: tests (5, ubuntu-latest, py312)
- GitHub Check: tests (5, ubuntu-latest, py311)
- GitHub Check: tests (7, ubuntu-latest, py311)
- GitHub Check: tests (4, macos-latest, py313)
- GitHub Check: tests (4, windows-2022, py313)
- GitHub Check: tests (4, ubuntu-latest, py313)
- GitHub Check: tests (3, ubuntu-latest, py313)
- GitHub Check: tests (3, windows-2022, py313)
- GitHub Check: tests (2, macos-latest, py312)
- GitHub Check: tests (4, ubuntu-latest, py312)
- GitHub Check: tests (3, ubuntu-latest, py311)
- GitHub Check: tests (4, ubuntu-latest, py311)
- GitHub Check: tests (2, ubuntu-latest, py312)
- GitHub Check: tests (3, macos-latest, py312)
- GitHub Check: tests (3, ubuntu-latest, py312)
- GitHub Check: tests (1, ubuntu-latest, py311)
- GitHub Check: tests (1, ubuntu-latest, py312)
- GitHub Check: tests (1, windows-2022, py312)
- GitHub Check: tests (2, macos-latest, py313)
- GitHub Check: tests (2, windows-2022, py313)
- GitHub Check: tests (2, windows-2022, py312)
- GitHub Check: tests (3, windows-2022, py312)
- GitHub Check: tests (1, windows-2022, py313)
- GitHub Check: tests (2, ubuntu-latest, py311)
- GitHub Check: tests (1, macos-latest, py312)
- GitHub Check: tests (1, ubuntu-latest, py313)
- GitHub Check: tests (2, ubuntu-latest, py313)
🔇 Additional comments (1)
tests/tests.py (1)
861-870: LGTM: test exercises source-cache bind with explicit apptainer argsCovers the regression path where user-provided binds previously disabled the source-cache mount.
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (1)
src/snakemake/deployment/singularity.py (1)
132-139: Unconditional source-cache bind and fixed spacing/quoting — LGTMThis addresses the bug (bind now added regardless of extra args) and resolves the earlier whitespace/quoting issue when appending --bind.
🧹 Nitpick comments (3)
src/snakemake/deployment/singularity.py (3)
136-136: Use f-string conversion flag instead of repr() (RUF010)Satisfy Ruff and align with common style.
- args += f" --bind {repr(source_cache_path)}" + args += f" --bind {source_cache_path!r}"
135-135: Prefer isdir() to avoid binding a non-directory pathSlightly safer and semantically tighter.
- if os.path.exists(source_cache_path): + if os.path.isdir(source_cache_path):
138-139: Nit: log the path with !r for clarity/consistencyMatches the quoting used elsewhere.
- logger.debug( - f"Source cache directory {source_cache_path} does not exist, skipping bind mount" + logger.debug( + f"Source cache directory {source_cache_path!r} does not exist, skipping bind mount"
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
src/snakemake/deployment/singularity.py(1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.py
⚙️ CodeRabbit configuration file
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
Files:
src/snakemake/deployment/singularity.py
🧬 Code graph analysis (1)
src/snakemake/deployment/singularity.py (2)
src/snakemake/workflow.py (1)
source_cache_path(280-290)src/snakemake/common/__init__.py (1)
get_appdirs(118-124)
🪛 Ruff (0.12.2)
src/snakemake/deployment/singularity.py
136-136: Use explicit conversion flag
Replace with conversion flag
(RUF010)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (55)
- GitHub Check: tests (10, windows-2022, py312)
- GitHub Check: tests (9, windows-2022, py313)
- GitHub Check: tests (10, ubuntu-latest, py313)
- GitHub Check: tests (10, windows-2022, py313)
- GitHub Check: tests (8, ubuntu-latest, py311)
- GitHub Check: tests (10, ubuntu-latest, py311)
- GitHub Check: tests (9, windows-2022, py312)
- GitHub Check: tests (10, ubuntu-latest, py312)
- GitHub Check: tests (9, ubuntu-latest, py312)
- GitHub Check: tests (9, ubuntu-latest, py313)
- GitHub Check: tests (7, ubuntu-latest, py313)
- GitHub Check: tests (9, ubuntu-latest, py311)
- GitHub Check: tests (8, windows-2022, py312)
- GitHub Check: tests (7, ubuntu-latest, py312)
- GitHub Check: tests (8, windows-2022, py313)
- GitHub Check: tests (7, windows-2022, py313)
- GitHub Check: tests (8, ubuntu-latest, py313)
- GitHub Check: tests (7, ubuntu-latest, py311)
- GitHub Check: tests (8, ubuntu-latest, py312)
- GitHub Check: tests (7, windows-2022, py312)
- GitHub Check: tests (2, ubuntu-latest, py311)
- GitHub Check: tests (6, ubuntu-latest, py311)
- GitHub Check: tests (6, windows-2022, py313)
- GitHub Check: tests (6, ubuntu-latest, py312)
- GitHub Check: tests (6, ubuntu-latest, py313)
- GitHub Check: tests (5, ubuntu-latest, py312)
- GitHub Check: tests (5, windows-2022, py313)
- GitHub Check: tests (6, windows-2022, py312)
- GitHub Check: tests (5, windows-2022, py312)
- GitHub Check: tests (4, ubuntu-latest, py311)
- GitHub Check: tests (2, macos-latest, py313)
- GitHub Check: tests (4, windows-2022, py312)
- GitHub Check: tests (5, ubuntu-latest, py313)
- GitHub Check: tests (4, macos-latest, py313)
- GitHub Check: tests (4, windows-2022, py313)
- GitHub Check: tests (4, ubuntu-latest, py312)
- GitHub Check: tests (4, ubuntu-latest, py313)
- GitHub Check: tests (5, ubuntu-latest, py311)
- GitHub Check: tests (3, ubuntu-latest, py311)
- GitHub Check: tests (3, ubuntu-latest, py312)
- GitHub Check: tests (3, windows-2022, py313)
- GitHub Check: tests (3, ubuntu-latest, py313)
- GitHub Check: tests (2, windows-2022, py313)
- GitHub Check: tests (3, windows-2022, py312)
- GitHub Check: tests (2, windows-2022, py312)
- GitHub Check: tests (2, macos-latest, py312)
- GitHub Check: tests (2, ubuntu-latest, py313)
- GitHub Check: tests (1, ubuntu-latest, py313)
- GitHub Check: tests (2, ubuntu-latest, py312)
- GitHub Check: tests (1, macos-latest, py313)
- GitHub Check: tests (1, ubuntu-latest, py312)
- GitHub Check: tests (1, windows-2022, py313)
- GitHub Check: tests (1, macos-latest, py312)
- GitHub Check: tests (1, windows-2022, py312)
- GitHub Check: tests (1, ubuntu-latest, py311)
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
tests/tests.py (1)
834-843: LGTM! Test correctly exercises the source-cache mounting scenario.The test follows the established pattern and appropriately reproduces the bug scenario where
apptainer_argsare provided alongside source-cache usage.Consider adding an additional test case without
apptainer_argsto ensure the fix doesn't break the default case. For example:@skip_on_windows @apptainer @connected def test_singularity_source_cache_no_args(): run( dpath("test_singularity_source_cache"), deployment_method={DeploymentMethod.APPTAINER}, )This would provide better coverage by testing both scenarios (with and without extra args).
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
tests/tests.py(1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.py
⚙️ CodeRabbit configuration file
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
Files:
tests/tests.py
🧬 Code graph analysis (1)
tests/tests.py (1)
tests/common.py (2)
run(152-503)dpath(33-36)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (45)
- GitHub Check: tests (7, ubuntu-latest, py312)
- GitHub Check: tests (10, windows-2022, py313)
- GitHub Check: tests (8, ubuntu-latest, py313)
- GitHub Check: tests (10, ubuntu-latest, py313)
- GitHub Check: tests (10, ubuntu-latest, py311)
- GitHub Check: tests (10, ubuntu-latest, py312)
- GitHub Check: tests (8, windows-2022, py313)
- GitHub Check: tests (8, ubuntu-latest, py312)
- GitHub Check: tests (7, ubuntu-latest, py313)
- GitHub Check: tests (9, ubuntu-latest, py311)
- GitHub Check: tests (9, macos-latest, py313)
- GitHub Check: tests (7, windows-2022, py313)
- GitHub Check: tests (4, ubuntu-latest, py312)
- GitHub Check: tests (9, windows-2022, py313)
- GitHub Check: tests (7, ubuntu-latest, py311)
- GitHub Check: tests (6, macos-latest, py313)
- GitHub Check: tests (8, ubuntu-latest, py311)
- GitHub Check: tests (6, ubuntu-latest, py312)
- GitHub Check: tests (9, ubuntu-latest, py312)
- GitHub Check: tests (9, ubuntu-latest, py313)
- GitHub Check: tests (5, windows-2022, py313)
- GitHub Check: tests (6, windows-2022, py313)
- GitHub Check: tests (4, ubuntu-latest, py313)
- GitHub Check: tests (4, windows-2022, py313)
- GitHub Check: tests (6, ubuntu-latest, py311)
- GitHub Check: tests (5, ubuntu-latest, py312)
- GitHub Check: tests (3, macos-latest, py313)
- GitHub Check: tests (5, ubuntu-latest, py313)
- GitHub Check: tests (6, ubuntu-latest, py313)
- GitHub Check: tests (4, ubuntu-latest, py311)
- GitHub Check: tests (2, ubuntu-latest, py311)
- GitHub Check: tests (2, macos-latest, py313)
- GitHub Check: tests (5, ubuntu-latest, py311)
- GitHub Check: tests (3, ubuntu-latest, py313)
- GitHub Check: tests (2, windows-2022, py313)
- GitHub Check: tests (3, windows-2022, py313)
- GitHub Check: tests (3, ubuntu-latest, py311)
- GitHub Check: tests (3, ubuntu-latest, py312)
- GitHub Check: tests (2, ubuntu-latest, py312)
- GitHub Check: tests (1, ubuntu-latest, py313)
- GitHub Check: tests (1, ubuntu-latest, py311)
- GitHub Check: tests (2, ubuntu-latest, py313)
- GitHub Check: tests (1, macos-latest, py313)
- GitHub Check: tests (1, windows-2022, py313)
- GitHub Check: tests (1, ubuntu-latest, py312)
cademirch
left a comment
There was a problem hiding this comment.
Looks good to me. Its not clear to me why the len(args)==0 check was there in the first place, it seems like that makes it impossible to use the source cache and specify your own apptainer args.
🤖 I have created a release *beep* *boop* --- ## [9.11.7](v9.11.6...v9.11.7) (2025-10-01) ### Bug Fixes * Add type annotations in logging module ([#3757](#3757)) ([63404b2](63404b2)) * source_path not mounted ([#3738](#3738)) ([86e30b3](86e30b3)) ### Documentation * mastodon announcements ([#3723](#3723)) ([7da1b18](7da1b18)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
fixes snakemake#3737 ### QC <!-- Make sure that you can tick the boxes below. --> * [x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [X] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Source cache is now mounted in Singularity/Apptainer runs whenever the cache directory exists, even if extra container arguments are provided; a debug message is logged when the cache is missing. * **Tests** * Added an integration test validating source-cache binding alongside custom bind arguments. * Added a sample workflow rule to verify copying a source file to an output inside a container. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: Cade Mirchandani <[email protected]>
🤖 I have created a release *beep* *boop* --- ## [9.11.7](snakemake/snakemake@v9.11.6...v9.11.7) (2025-10-01) ### Bug Fixes * Add type annotations in logging module ([snakemake#3757](snakemake#3757)) ([63404b2](snakemake@63404b2)) * source_path not mounted ([snakemake#3738](snakemake#3738)) ([86e30b3](snakemake@86e30b3)) ### Documentation * mastodon announcements ([snakemake#3723](snakemake#3723)) ([7da1b18](snakemake@7da1b18)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
fixes #3737
QC
docs/) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).Summary by CodeRabbit
Bug Fixes
Tests