Commit bd94dc1
fix: Confusion with Overriding input After Snakemake Modularization (#3714)
will fix #3713
## 1: Support recursive module modification via `WorkflowModifier`
To make workflows recursively apply modifiers like path prefix and rule
name changes across nested modules, I introduced the following changes:
- src/snakemake/ruleinfo.py:
- Made `RuleInfo` recursively apply `parent_modifier` to adjust paths
and rule names.
- src/snakemake/path_modifier.py:
- Added `inner_modifier` to support chained path transformations in
nested modules.
- src/snakemake/modules.py:
- Modified `get_name_modifier_func` and
`WorkflowModifier.modify_rulename` to support recursive rule renaming.
- Replaced `WorkflowModifier.skip_rule` with `avail_rulename` to ensure
that `rule_whitelist` and `rule_exclude_list` are respected at every
module level.
## 2: Make multiple use statements with specific_rule more reliable and
strict
My understanding is that `rule.name` should be unique in a workflow.
Therefore, a second `use rule from ... with ...` should only be allowed
if the rule was previously imported with the same name via a wildcard
(`use * from ...`) and is now being refined.
- to clarify:
- A given rule from a module can be used **multiple times** in the same
workflow, as long as each usage assigns it a **unique name**.
- However, using the **same rule name** more than once is **not
allowed**, to prevent accidental overwrites, just like any rule rule
defined **out of** module **cannot** be overwrite.
- That’s the restriction behind the “only once” statement -- it applies
per final rule name, not per source rule.
- src/snakemake/rules.py:
- The old logic stored rule dependencies as objects. If a rule was
modified afterward, those changes wouldn't propagate.
- Now, rule dependencies are resolved dynamically by name, ensuring
correct updated rule with parameters modified.
- src/snakemake/modules.py:
- Stricter checks for rules overwriting. `ModuleInfo.use_rules` now
`allow_rule_overwrite` for specific_rule only if it was previously
imported via `use * from ...` and not already customized.
- src/snakemake/workflow.py:
- Adjusted to work with the new `WorkflowModifier`.
## Other changes:
- Simplify `WorkflowModifier.__init__`. The module’s `__name__` is now
assigned as `WorkflowModifier.namespace`, making workflow development
and debugging easier.
### QC
* [x] The PR contains a test case for the changes or the changes are
already covered by an existing test case.
* [x] The documentation (`docs/`) is updated to reflect the changes or
this is not necessary (e.g. if the change does neither modify the
language nor the behavior or functionalities of Snakemake).
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Multi-stage path modifiers, a public module registry, and improved
name/path/wrapper modifier lifecycle for more predictable module
imports.
* **Bug Fixes**
* Prevents unintended duplicate rule creation from wildcard/name
modifications and enforces explicit overwrite semantics; tighter
path-replacement guards and more reliable dependency resolution.
* **Tests**
* Updated fixtures for renamed, aliased and nested modules/rules to
reflect modifier behavior changes.
* **Documentation**
* Expanded module-import and rule-customization guidance with examples
and conflict-resolution advice.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Johannes Köster <[email protected]>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>1 parent 3ffd8e1 commit bd94dc1
13 files changed
Lines changed: 295 additions & 160 deletions
File tree
- docs/snakefiles
- src/snakemake
- tests
- test_module_nested
- test_modules_all
- module-test
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
128 | 128 | | |
129 | 129 | | |
130 | 130 | | |
131 | | - | |
| 131 | + | |
132 | 132 | | |
133 | 133 | | |
134 | 134 | | |
| |||
160 | 160 | | |
161 | 161 | | |
162 | 162 | | |
163 | | - | |
| 163 | + | |
164 | 164 | | |
165 | 165 | | |
166 | 166 | | |
167 | 167 | | |
168 | 168 | | |
169 | 169 | | |
170 | | - | |
| 170 | + | |
171 | 171 | | |
172 | | - | |
173 | | - | |
174 | | - | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
175 | 175 | | |
176 | 176 | | |
177 | 177 | | |
| |||
189 | 189 | | |
190 | 190 | | |
191 | 191 | | |
192 | | - | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
193 | 196 | | |
194 | 197 | | |
195 | | - | |
196 | | - | |
197 | 198 | | |
198 | | - | |
199 | | - | |
200 | | - | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
201 | 238 | | |
202 | 239 | | |
203 | 240 | | |
| |||
209 | 246 | | |
210 | 247 | | |
211 | 248 | | |
212 | | - | |
| 249 | + | |
213 | 250 | | |
214 | 251 | | |
215 | 252 | | |
| |||
289 | 326 | | |
290 | 327 | | |
291 | 328 | | |
292 | | - | |
| 329 | + | |
293 | 330 | | |
294 | 331 | | |
295 | 332 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2944 | 2944 | | |
2945 | 2945 | | |
2946 | 2946 | | |
| 2947 | + | |
| 2948 | + | |
| 2949 | + | |
| 2950 | + | |
| 2951 | + | |
| 2952 | + | |
2947 | 2953 | | |
2948 | 2954 | | |
2949 | 2955 | | |
| |||
0 commit comments