Snakemake version
9.11.2
Describe the bug
Running rules in containers with additional --apptainer-args doesn't mount the snakemake cache correctly, which causes an error when using workflow.source_path.
Minimal example
Snakefile:
rule all:
input:
expand("results/{name}.txt", name=["A", "B"])
rule A:
output:
"results/A.txt"
shell:
"touch {output}"
rule B:
output:
"results/B.txt"
params:
B = workflow.source_path("resources/B.txt")
singularity:
"docker://bash:5.3.3"
shell:
"cp {params.B} {output}"Command-line:
snakemake -c1 --sdm apptainer --apptainer-args "-B /tmp/example:/tmp/example"
Logs
Assuming unrestricted shared filesystem usage.
None
host: bif02.ircm.priv
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job stats:
job count
----- -------
A 1
B 1
all 1
total 3
Select jobs to execute...
Execute 1 jobs...
[Wed Sep 10 14:00:21 2025]
localrule A:
output: results/A.txt
jobid: 1
reason: Missing output files: results/A.txt
resources: tmpdir=/tmp
[Wed Sep 10 14:00:21 2025]
Finished jobid: 1 (Rule: A)
1 of 3 steps (33%) done
Select jobs to execute...
Execute 1 jobs...
[Wed Sep 10 14:00:21 2025]
localrule B:
output: results/B.txt
jobid: 2
reason: Missing output files: results/B.txt; Params have changed since last execution: Union of exclusive params before and now across all output: before: '/home/reuletg/.cache/snakemake/snakemake/source-cache/runtime-cache/tmpus2orkyd/file/stockage/Laboratoires_BIF/BIF_DATA/reuletg/fix-snakemake-workflow-source-path/B/resources/B.txt' now: '/home/reuletg/.cache/snakemake/snakemake/source-cache/runtime-cache/tmpnzc5_1jq/file/stockage/Laboratoires_BIF/BIF_DATA/reuletg/fix-snakemake-workflow-source-path/B/resources/B.txt'
resources: tmpdir=/tmp
Activating singularity image /usr/local/tools/apptainer/cache/cd22e171670c63e6b1e458f41cae1016.simg
RuleException:
CalledProcessError in file "/stockage/Laboratoires_BIF/BIF_DATA/reuletg/fix-snakemake-workflow-source-path/B/Snakefile", line 25:
Command ' singularity exec --home '/stockage/Laboratoires_BIF/BIF_DATA/reuletg/fix-snakemake-workflow-source-path/B' -B /tmp/example:/tmp/example /usr/local/tools/apptainer/cache/cd22e171670c63e6b1e458f41cae1016.simg bash -c 'set -euo pipefail; cp /home/reuletg/.cache/snakemake/snakemake/source-cache/runtime-cache/tmpnzc5_1jq/file/stockage/Laboratoires_BIF/BIF_DATA/reuletg/fix-snakemake-workflow-source-path/B/resources/B.txt results/B.txt'' returned non-zero exit status 1.
[Wed Sep 10 14:00:21 2025]
Error in rule B:
message: None
jobid: 2
output: results/B.txt
shell:
cp /home/reuletg/.cache/snakemake/snakemake/source-cache/runtime-cache/tmpnzc5_1jq/file/stockage/Laboratoires_BIF/BIF_DATA/reuletg/fix-snakemake-workflow-source-path/B/resources/B.txt results/B.txt
(command exited with non-zero exit code)
Shutting down, this might take some time.
Exiting because a job execution failed. Look below for error messages
[Wed Sep 10 14:00:21 2025]
Error in rule B:
message: None
jobid: 2
output: results/B.txt
shell:
cp /home/reuletg/.cache/snakemake/snakemake/source-cache/runtime-cache/tmpnzc5_1jq/file/stockage/Laboratoires_BIF/BIF_DATA/reuletg/fix-snakemake-workflow-source-path/B/resources/B.txt results/B.txt
(command exited with non-zero exit code)
Complete log(s): /stockage/Laboratoires_BIF/BIF_DATA/reuletg/fix-snakemake-workflow-source-path/B/.snakemake/log/2025-09-10T140021.484272.snakemake.log
WorkflowError:
At least one job did not complete successfully.
Additional context
The issue comes from:
|
if len(args) == 0: |
|
source_cache_path = os.path.join( |
|
get_appdirs().user_cache_dir, "snakemake/source-cache" |
|
) |
|
if os.path.exists(source_cache_path): |
|
args += "--bind " + source_cache_path |
|
else: |
|
logger.debug( |
|
f"Source cache directory {source_cache_path} does not exist, skipping bind mount" |
|
) |
|
|
If any args are given on the command-line with --apptainer-args, the snakemake source-cache is not added to the bind mounts.
It should be a simple fix, I don't think the if statement is needed, I will open a PR with a fix.
Snakemake version
9.11.2
Describe the bug
Running rules in containers with additional
--apptainer-argsdoesn't mount the snakemake cache correctly, which causes an error when usingworkflow.source_path.Minimal example
Snakefile:
Command-line:
snakemake -c1 --sdm apptainer --apptainer-args "-B /tmp/example:/tmp/example"Logs
Additional context
The issue comes from:
snakemake/src/snakemake/deployment/singularity.py
Lines 132 to 142 in fcf9bcf
If any args are given on the command-line with
--apptainer-args, the snakemake source-cache is not added to the bind mounts.It should be a simple fix, I don't think the
ifstatement is needed, I will open a PR with a fix.