Comparing changes

Fixes #3592 ### QC  * [ ] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [ ] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).  ## Summary by CodeRabbit - **Bug Fixes** - Environment variables within profile configuration values are now automatically expanded, ensuring correct values are used in configurations.

### Description  Closes #3422 ### QC  * [x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).  ## Summary by CodeRabbit ## Summary by CodeRabbit - **New Features** - Introduced a new property to evaluate non-local execution and dry run scenarios. - **Bug Fixes** - Refined task grouping behavior to ensure accurate assignment based on the updated execution context. These updates aim to provide more consistent and predictable workflow behavior across varied execution modes.  Co-authored-by: Tadas Bareikis <[email protected]>

Similar to #3542. Now the current version of PuLP is [3.2.1](https://github.com/coin-or/pulp/releases/tag/3.2.1). Nothing in the changes [from 3.1.1 to 3.2.1](coin-or/pulp@3.1.1...3.2.1) seems like it should cause any obvious problems for Snakemake, so let’s try to keep up with the latest PuLP releases. ### QC  * [x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).

This PR closes #2468, by replacing RefResolver functionality with the referencing library. ### QC  * [x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).  ## Summary by CodeRabbit - **Refactor** - Modernized the data validation process by adopting the latest features of the JSON Schema Draft 2020-12 specification for improved reliability and consistency. - Added schema versioning information to existing schema definitions for enhanced validation and interoperability. - **Style** - Enhanced the formatting of warning messages for clearer and more readable notifications.

As described in #348, the current notebook integration has some downsides, and they are due to limitations in the `nbconvert` tool. [`papermill`](https://papermill.readthedocs.io/en/latest/index.html) is a more feature-rich alternative. In particular, it outputs the executed notebook incrementally for progress tracking, and outputs the notebook (with errors) even when it fails. ### QC * [x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake). I didn't see `nbconvert` anywhere in the docs, so I think this was always an implementation detail.  ## Summary by CodeRabbit - **Tests** - Added new test functions for verifying Jupyter notebook execution in various scenarios. - **Documentation** - Updated guidance to clarify headless notebook execution and expectations during errors. - **Refactor** - Enhanced error handling and improved command execution logic for more reliable Jupyter notebook processing.  --------- Co-authored-by: Christian Meesters <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: Filipe G. Vieira <[email protected]> Co-authored-by: Johannes Köster <[email protected]>

### QC  * [x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).  ## Summary by CodeRabbit - **Enhancements** - Improved warning messages when conda environment installation fails, now including detailed error output for easier troubleshooting. - Toggle controls in reports now initialize to "yes" when specified and more accurately reflect the current state. - **Improvements** - Toggle components now use a dedicated default value for initial state, ensuring consistency. - **Style** - Removed unnecessary debugging output from toggle controls.

…nsistency & Ease (#3605) Updated docs with recent version of the wrapper for the rule samtools_index and used this rule because it is easy to follow and also consistent with the previous step, automatic deployment of software dependencies. Fixes #3585  ## Summary by CodeRabbit - **Documentation** - Updated the tutorial example to demonstrate the use of a samtools indexing wrapper instead of a BWA mapping wrapper. - Revised input, output, log, params, and wrapper directives in the example for clarity and accuracy. - Added a conda environment specification for samtools in the example.  Co-authored-by: rohan-ibn-tariq <[email protected]>

…ation, and Reporting with Snakemake) (#3606) Fixed caption file names and report directive path for tutorial2 step6(Reporting). As following the tutorial example with current file names or report directive path was causing obvious issue(s). Fixes #3588  ## Summary by CodeRabbit - **Documentation** - Updated tutorial instructions and captions to reference more specific report filenames for R and Python plot rules. - Revised global report directive to reflect the new report file path. - Improved consistency in file naming throughout the tutorial.  Co-authored-by: rohan-ibn-tariq <[email protected]>

… Installing The Tools Manually (#3607) Updated requirement section docs in the tutorial, to guide the user to wait for preferred ways of installation of required tools to appear in the following sections and not to install the tools one by one by yourself through the terminal manually etc. This was important as some users could do that as the warning may be hidden as in the original version of docs and secondly had grammatical mistake. Fixes #3582  ## Summary by CodeRabbit - **Documentation** - Improved setup instructions in the tutorial to more clearly advise users to wait for guided installation steps, enhancing clarity and reducing confusion.  Co-authored-by: rohan-ibn-tariq <[email protected]>

…3608) Added snakemake command to execute the rule(plot_with_python) with notebook editing option in step 3 of tutorial 2 pertaining to snakemake for the sake of consistency with the previous step. Fixes #3587.  ## Summary by CodeRabbit - **Documentation** - Improved the tutorial by providing a clear, explicit command example for editing the Altair plot notebook interactively, replacing a vague instruction with a concrete step.  Co-authored-by: rohan-ibn-tariq <[email protected]>

I am using SnakeMake with wildcards for a very large number of files. I observed that executimg `snakemake --quiet rules` would still leave blank lines corresponding to the space where the rules would have been. ![image](https://github.com/user-attachments/assets/528e72d7-f6cd-4096-82df-5a10b3d5068f) I am not fully fammiliar with the snakemake logging setup but i narrowed down the problem to the `emit` mehod of `ColorizingTextHandler`. My solution was to check for an empty `formatted_message`. Our large number of files also use a corresponding number of temp files which results in thousands of print messages about which temporary files would be remove. I purpose adding `LogEvent.JOB_INFO` to the temporary file messages in order to allow quieting these messages. ### QC  * [x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).  ## Summary by CodeRabbit - **Bug Fixes** - Improved logging to prevent empty or "None" messages from appearing in the output. - Refined logging behavior to respect quiet mode settings, reducing unnecessary log messages during temporary file removal and job scheduling progress updates. - Enhanced control over shell command and progress log visibility based on specific quietness flags for a clearer user experience.  --------- Co-authored-by: Christoffer <[email protected]> Co-authored-by: Cade Mirchandani <[email protected]>

…emote job (#3625) ### Description  ### QC  * [x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).  ## Summary by CodeRabbit - **New Features** - Added support for specifying a pre-command to be executed before each remote job when running workflows with remote execution.

### Description Fixes #3570 : Overwriting params for rule inheritance did not work properly when the parent rule did not define a params directive at all. ### QC * [x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).  ## Summary by CodeRabbit - **Bug Fixes** - Improved handling of rule parameters to prevent errors when parameters are missing or set to None. - **Tests** - Added new tests to verify correct behavior when inheriting and overriding rule parameters. - Included expected output files to validate test results.

Removed wms monitor arg and updated monitoring docs. closes #3620 #3619 ### QC  * [x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).  ## Summary by CodeRabbit ## Summary by CodeRabbit - **Documentation** - Updated monitoring documentation to introduce logger plugins as the new monitoring mechanism, replacing previous references to Panoptes and the `--wms-monitor` flag. Added instructions for using logger plugins and linked to the plugin catalog. - **Bug Fixes** - Removed obsolete `--wms-monitor` and `--wms-monitor-arg` command-line options and related configuration support.

Issue #3192 is related to a typo in `CondaEnvDirSpec.__eq__`. This PR fixes this issue and add a test case. Fixes #3192 ### QC  * [X] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [X] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).  ## Summary by CodeRabbit ## Summary by CodeRabbit - **Bug Fixes** - Improved the handling of conda environment directory comparisons to ensure correct behavior when matching environment paths. - **Tests** - Added a new test workflow to verify conda environment handling, including sample input, expected output, and supporting scripts. - Introduced a dedicated test to validate the new conda environment behavior.

…3626) ### Description  ### QC  * [x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).

@Hocnonsense

### Description Follow-up change on merged PR #3624 , inspired by suggested PR #3496 (probs to @Hocnonsense ). ### QC * [x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).  ## Summary by CodeRabbit - **Bug Fixes** - Improved handling of rule parameters to prevent errors when parameters are missing.

🤖 I have created a release *beep* *boop* --- ## [9.6.0](v9.5.1...v9.6.0) (2025-06-16) ### Features * Prefer papermill to nbconvert ([#2857](#2857)) ([4263b03](4263b03)) ### Bug Fixes * DeprecationWarning when using snakemake.utils.validate ([#3420](#3420)) ([cf72427](cf72427)) * display group jobs on dryrun ([#3435](#3435)) ([3bebef4](3bebef4)) * expandvars for special profile keys ([#3597](#3597)) ([4020188](4020188)) * fix bug causing --precommand to not being executed before each remote job ([#3625](#3625)) ([e59d125](e59d125)) * improved toggle switch behavior in reports ([#3623](#3623)) ([0c4bd23](0c4bd23)) * pass envvars defined via the envvars directive to remote jobs ([#3626](#3626)) ([d4890b4](d4890b4)) * remove wms arg, update logging cli docs ([#3622](#3622)) ([3a9a5ac](3a9a5ac)) * typo in CondaEnvDirSpec.__eq__ (issue [#3192](#3192)) ([#3613](#3613)) ([f4c107f](f4c107f)) * Unclear handling of params overriding with inheritance ([#3624](#3624)) ([077ac4a](077ac4a)) ### Documentation * Added snakemake command to execute the rule plot_with_python ([#3608](#3608)) ([bd99c11](bd99c11)) * Updated Reporting Section of The Tutorial(Interaction, Visualization, and Reporting with Snakemake) ([#3606](#3606)) ([91e90ba](91e90ba)) * Updated Requirement Section of The Tutorial With Warning of Not Installing The Tools Manually ([#3607](#3607)) ([3bd114b](3bd114b)) * Updated Wrapper Version in Tutorial and Used Simple Rule For Consistency & Ease ([#3605](#3605)) ([b3bcc21](b3bcc21)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comparing changes

Open a pull request

Commits on May 30, 2025

Commits on Jun 1, 2025

Commits on Jun 3, 2025

Commits on Jun 11, 2025

Commits on Jun 12, 2025

Commits on Jun 13, 2025

Commits on Jun 16, 2025

This comparison is taking too long to generate.

Uh oh!