Comparing changes

…t retrievable (#3892)  ### QC  * [x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).  ## Summary by CodeRabbit ## Release Notes * **Bug Fixes** * Improved error messages for Git-sourced workflow files with enhanced diagnostics and troubleshooting guidance * Enhanced error reporting with clearer path information and better traceability for file access failures ✏️ Tip: You can customize this high-level summary in your review settings.

I struggled to pass in nested config (the first thing I tried was dot format, like foo.bar=42) until I found [this SO post](https://stackoverflow.com/questions/64271392/overwrite-nested-config-options-on-command-line). ### QC  * [x] N/A The PR contains a test case for the changes or the changes are already covered by an existing test case. * [x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).  ## Summary by CodeRabbit * **Documentation** * Enhanced help text for the --config option with improved guidance on nested value formatting and practical usage examples to assist users in proper configuration definition. ✏️ Tip: You can customize this high-level summary in your review settings.

Currently there is a single global `LoggerManager` instance, with its `setup()` method called from `SnakemakeApi`'s constructor. This is a potential source of issues if the global instance gets used multiple times in the same Python session, as happens when running the test suite. It's also not very necessary, as the global is only used in a couple of places (`api.py` and `workflow.py`), and there's not really any setup or configuration that actually needs to be shared (a couple of exceptions are mentioned below). This PR removes the global instance and instead creates a new one attached to each `SnakemakeApi` instance. This instance is also now passed explicitly to `Workflow`'s constructor. Because the setup now happens when the `LoggerManager` is created, I removed the `setup()` function and merged its contents into `__init__()`. I also refactored and simplified some of the code previously in `setup()`, and also moved the initial creation of plugin log handlers from `SnakemakeApi.setup_logger()` to `LoggerManager.__init__()`. Edit: Also removed the `cleanup_logfile()` method and instead remove all handlers from the logger in `stop()`. ### Potential issues #### No tests Currently there are no tests for most of this behavior, that is fixed in #3848. I have personally tested that those tests still succeed with these changes, but probably best to merge that PR first. #### Global logger There is still a single global `Logger` instance, which gets (mostly) reset and then set up again for each `LoggerManager` instance. Ideally each `LoggerManager` would create its own logger, however this is much more difficult to refactor because the global logger is used extensively throughout the codebase. I'll leave that to a separate PR/issue. #### `OutputSettings.keep_logger` Setup and teardown of the global `LoggerManager` was previously skipped if `OutputSettings.keep_logger` was True. As far as I can tell this was never the case - the dataclass field's default value is False, I checked all instances I could find of the constructor being called and the only case that sets the field is in `snakemake/cli.py:create_output_settings()`, where it is always set to False ([link](https://github.com/snakemake/snakemake/blob/12ce6c6cb854867cf90255c4119b73614a1ec58a/src/snakemake/cli.py#L1923)). I removed the attribute from `OutputSettings`. If we want to optionally allow for similar behavior, I could add an argument to `SnakemakeApi`'s constructor that accepts an existing instance. #### Default stream handler level In current `LoggerManager.setup()`: ```python3 class LoggerManager: def setup(...): ... # Skip plugin handlers if requested and return early if settings.skip_plugin_handlers or not handlers: handler = self._default_streamhandler() if settings.log_level_override is not None: handler.setLevel(settings.log_level_override) self.logger.addHandler(handler) ... return ``` I removed this if statement because most of the code was duplicated below (some of it is conditional on no plugin handlers outputting to stream, which is always the case if there are no handlers or plugins are skipped). The only thing that is different is that the default stream handler's level is set based on `log_level_override`, which doesn't happen otherwise. I'm pretty confident this has no effect because the `Logger` the handler is attached to is set to the same level.  ## Summary by CodeRabbit * **Refactor** * Centralized logging into a manager with plugin-capable initialization, unified lifecycle and cleanup; logging context is now passed into workflow execution and logfile handling is managed centrally. * **Chores** * Marked the previous keep_logger setting as deprecated and removed its use during output-settings creation to simplify logging configuration. ✏️ Tip: You can customize this high-level summary in your review settings.  --------- Co-authored-by: Cade Mirchandani <[email protected]>

Pixi install github action is failing with "failed to parse pypi name mapping" errors likely due to rate limiting when 30+ jobs are kicked off nearly simultaneously I tested this fix on #3820 since the tests kept failing due to the `pixi` install action failing. After committing this change, [the actions ran successfully](https://github.com/snakemake/snakemake/actions/runs/20684893321). In [this failing run's](https://github.com/snakemake/snakemake/actions/runs/20682879051/job/59383597583#step:3:3261) debug logs we see: ``` pixi install -e py311 [...] WARN resolve_conda{group=py313 platform=win-64}: reqwest_retry::middleware: Retry attempt #1. Sleeping 1.225245051s before the next attempt Error: × failed to parse pypi name mapping ├─▶ error decoding response body ╰─▶ expected value at line 1 column 1 ``` This warning is repeated many times until finally pixi stops retrying - this is what suggested to me that some sort of rate limit was the issue. One downside is that this does make the CI take a bit longer to run. We could consider using the `cache` feature of the pixi action. And turning up the max-parallel, or reducing the number of test-groups ### QC  * [ ] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [ ] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).  ## Summary by CodeRabbit * **Chores** * Updated development toolchain dependencies for improved build and test infrastructure. ✏️ Tip: You can customize this high-level summary in your review settings.

Hi everyone, The enumeration of available solvers with `pulp.listSolvers(onlyAvailable=True)` typically acquires license slots of all installed commercial solvers, which becomes problematic, if only a limited number of shared slots are available. pulp correctly releases the licenses immediately again, but it still repeatedly leads to friction. This PR avoids the pulp solver enumeration by using `pulp.getSolver(...).available()` to check for whether the chosen solver is available and only on failure queries the full list. The default solver choice from #3736 was updated to rely on `pulp.apis.LpSolverDefault`, which is compatible with "PULP_CBC_CMD" (if it is available). I also moved the pulp specific checking out of api.py entirely, so that all the solver choice is now handled by the new structure. With these changes, even if pulp is not installed, snakemake gracefully falls back to the greedy scheduler. ### QC  * [x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).  ## Summary by CodeRabbit * **Improvements** * Lazy, more robust solver discovery with improved default selection for ILP scheduling, reducing startup overhead and surprises. * Scheduler now detects when the configured solver is unavailable and automatically falls back to the greedy scheduler. * **New Features** * Exposes a solver-availability indicator for easier diagnostics and decisions. * **Bug Fixes** * Clearer warning when falling back, indicating missing solver support and required external solver dependencies. ✏️ Tip: You can customize this high-level summary in your review settings.  --------- Co-authored-by: Johannes Köster <[email protected]>

…oviders (#3643) ### Description  ### QC  * [x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).  ## Summary by CodeRabbit * **New Features** * Add Git-hosted source support for importing files/modules from GitHub/GitLab with token-aware access and on-disk caching. * **Improvements** * Propagate and improve runtime caching for hosted sources. * Safer secret-free path handling and centralized path validation/formatting. * Streamlined Conda environment spec detection and validation. * **Tests** * Added a GitHub-import integration test (skipped by default). ✏️ Tip: You can customize this high-level summary in your review settings.

…g command lines for spawned jobs (#3894)  will fix #3891 by force to create the .snakemake folder for the script ### QC  * [x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).  ## Summary by CodeRabbit * **Bug Fixes** * Enhanced robustness when executing shell commands with very long argument strings by ensuring proper directory initialization. * **Tests** * Added test coverage for handling extremely long command-line arguments in shadow mode. ✏️ Tip: You can customize this high-level summary in your review settings.

…Ds with fixed prefix length (#3889) This PR makes snakemake shorten report hashes to 16 characters and also makes `report_href()` use the 16 character long prefix which was not done before. Collisions are detected when shortening IDs and an error is raised directing users to open an issue to request a longer hash length. Fixes #3867.  ## Summary by CodeRabbit * **New Features** * Reports and generated links now use 16-character shortened file IDs for cleaner, more readable paths. * **Bug Fixes** * Added collision detection for shortened IDs; non-unique shortened IDs now produce a clear error with guidance to request longer hashes. * Removed prior silent-warning behavior in favor of explicit error handling. * **Style** * Minor formatting cleanup in generated report preambles (reduced extra blank lines). ✏️ Tip: You can customize this high-level summary in your review settings.

🤖 I have created a release *beep* *boop* --- ## [9.14.6](v9.14.5...v9.14.6) (2026-01-08) ### Bug Fixes * create local clone of git repos for source files from hosting providers ([#3643](#3643)) ([d2f8aba](d2f8aba)) * create potentially missing .snakemake folder in case of very long command lines for spawned jobs ([#3894](#3894)) ([4b431dd](4b431dd)) * make ilp solver enumeration lazy ([#3900](#3900)) ([30e1509](30e1509)) * Prevent broken report_href links by using deterministic report IDs with fixed prefix length ([#3889](#3889)) ([6d8f4d8](6d8f4d8)) * refactor LoggerManager setup and scope ([#3851](#3851)) ([f46d904](f46d904)) * yield proper error message in case a local git source file is not retrievable ([#3892](#3892)) ([ed79cae](ed79cae)) ### Documentation * explain how to pass nested config via CLI ([#3885](#3885)) ([9d8c539](9d8c539)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comparing changes

Open a pull request

Commits on Dec 19, 2025

Commits on Dec 23, 2025

Commits on Jan 5, 2026

Commits on Jan 7, 2026

Commits on Jan 8, 2026

This comparison is taking too long to generate.

Uh oh!