fix: expandvars for special profile keys#3597
Conversation
📝 WalkthroughWalkthroughThe change updates the profile configuration parser to ensure that all values in one-level and two-level dictionaries are processed with environment variable expansion using Changes
Assessment against linked issues
Note ⚡️ AI Code Reviews for VS Code, Cursor, WindsurfCodeRabbit now has a plugin for VS Code, Cursor and Windsurf. This brings AI code reviews directly in the code editor. Each commit is reviewed immediately, finding bugs before the PR is raised. Seamless context handoff to your AI code agent ensures that you can easily incorporate review feedback. Note ⚡️ Faster reviews with cachingCodeRabbit now supports caching for code and dependencies, helping speed up reviews. This means quicker feedback, reduced wait times, and a smoother review experience overall. Cached data is encrypted and stored securely. This feature will be automatically enabled for all accounts on May 30th. To opt out, configure 📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (1)
🧰 Additional context used📓 Path-based instructions (1)`**/*.py`: Do not try to improve formatting. Do not suggest type annotations for functions that are defined inside of functions or methods. Do not suggest type annotation of the `s...
⏰ Context from checks skipped due to timeout of 90000ms (36)
🔇 Additional comments (3)
✨ Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
|
Good idea! Maybe the profile docs should also be extended to explain that $-variables will be expanded. |
It is already documented, as pointed out in #3592 (comment). The issue fixed here is to make sure that |
🤖 I have created a release *beep* *boop* --- ## [9.6.0](v9.5.1...v9.6.0) (2025-06-16) ### Features * Prefer papermill to nbconvert ([#2857](#2857)) ([4263b03](4263b03)) ### Bug Fixes * DeprecationWarning when using snakemake.utils.validate ([#3420](#3420)) ([cf72427](cf72427)) * display group jobs on dryrun ([#3435](#3435)) ([3bebef4](3bebef4)) * expandvars for special profile keys ([#3597](#3597)) ([4020188](4020188)) * fix bug causing --precommand to not being executed before each remote job ([#3625](#3625)) ([e59d125](e59d125)) * improved toggle switch behavior in reports ([#3623](#3623)) ([0c4bd23](0c4bd23)) * pass envvars defined via the envvars directive to remote jobs ([#3626](#3626)) ([d4890b4](d4890b4)) * remove wms arg, update logging cli docs ([#3622](#3622)) ([3a9a5ac](3a9a5ac)) * typo in CondaEnvDirSpec.__eq__ (issue [#3192](#3192)) ([#3613](#3613)) ([f4c107f](f4c107f)) * Unclear handling of params overriding with inheritance ([#3624](#3624)) ([077ac4a](077ac4a)) ### Documentation * Added snakemake command to execute the rule plot_with_python ([#3608](#3608)) ([bd99c11](bd99c11)) * Updated Reporting Section of The Tutorial(Interaction, Visualization, and Reporting with Snakemake) ([#3606](#3606)) ([91e90ba](91e90ba)) * Updated Requirement Section of The Tutorial With Warning of Not Installing The Tools Manually ([#3607](#3607)) ([3bd114b](3bd114b)) * Updated Wrapper Version in Tutorial and Used Simple Rule For Consistency & Ease ([#3605](#3605)) ([b3bcc21](b3bcc21)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
<!--Add a description of your PR here--> Envvars on resources (e.g. `tmpdir`) are not expanded correctly (e.g. runtime). This PR fixes that, as well as expansion of envvars on profiles. For that reason, changes from PR #3597 were reverted. Tests were also added for both cases. ### QC <!-- Make sure that you can tick the boxes below. --> * [x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Configuration values and resource expressions now expand environment variables (e.g., $VAR) and user home (~), allowing dynamic paths and settings. * **Tests** * Test workflows updated to validate environment-variable and home-directory expansion in configs and resource handling. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes snakemake#3592 ### QC <!-- Make sure that you can tick the boxes below. --> * [ ] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [ ] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Bug Fixes** - Environment variables within profile configuration values are now automatically expanded, ensuring correct values are used in configurations. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
🤖 I have created a release *beep* *boop* --- ## [9.6.0](snakemake/snakemake@v9.5.1...v9.6.0) (2025-06-16) ### Features * Prefer papermill to nbconvert ([snakemake#2857](snakemake#2857)) ([4263b03](snakemake@4263b03)) ### Bug Fixes * DeprecationWarning when using snakemake.utils.validate ([snakemake#3420](snakemake#3420)) ([cf72427](snakemake@cf72427)) * display group jobs on dryrun ([snakemake#3435](snakemake#3435)) ([3bebef4](snakemake@3bebef4)) * expandvars for special profile keys ([snakemake#3597](snakemake#3597)) ([4020188](snakemake@4020188)) * fix bug causing --precommand to not being executed before each remote job ([snakemake#3625](snakemake#3625)) ([e59d125](snakemake@e59d125)) * improved toggle switch behavior in reports ([snakemake#3623](snakemake#3623)) ([0c4bd23](snakemake@0c4bd23)) * pass envvars defined via the envvars directive to remote jobs ([snakemake#3626](snakemake#3626)) ([d4890b4](snakemake@d4890b4)) * remove wms arg, update logging cli docs ([snakemake#3622](snakemake#3622)) ([3a9a5ac](snakemake@3a9a5ac)) * typo in CondaEnvDirSpec.__eq__ (issue [snakemake#3192](snakemake#3192)) ([snakemake#3613](snakemake#3613)) ([f4c107f](snakemake@f4c107f)) * Unclear handling of params overriding with inheritance ([snakemake#3624](snakemake#3624)) ([077ac4a](snakemake@077ac4a)) ### Documentation * Added snakemake command to execute the rule plot_with_python ([snakemake#3608](snakemake#3608)) ([bd99c11](snakemake@bd99c11)) * Updated Reporting Section of The Tutorial(Interaction, Visualization, and Reporting with Snakemake) ([snakemake#3606](snakemake#3606)) ([91e90ba](snakemake@91e90ba)) * Updated Requirement Section of The Tutorial With Warning of Not Installing The Tools Manually ([snakemake#3607](snakemake#3607)) ([3bd114b](snakemake@3bd114b)) * Updated Wrapper Version in Tutorial and Used Simple Rule For Consistency & Ease ([snakemake#3605](snakemake#3605)) ([b3bcc21](snakemake@b3bcc21)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
<!--Add a description of your PR here--> Envvars on resources (e.g. `tmpdir`) are not expanded correctly (e.g. runtime). This PR fixes that, as well as expansion of envvars on profiles. For that reason, changes from PR snakemake#3597 were reverted. Tests were also added for both cases. ### QC <!-- Make sure that you can tick the boxes below. --> * [x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Configuration values and resource expressions now expand environment variables (e.g., $VAR) and user home (~), allowing dynamic paths and settings. * **Tests** * Test workflows updated to validate environment-variable and home-directory expansion in configs and resource handling. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes #3592
Need to still add tests.
QC
docs/) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).Summary by CodeRabbit