fix: use proper default and fallback LP solver in scheduler settings (thanks @raffienficiaud)#3736
Merged
johanneskoester merged 4 commits intomainfrom Sep 16, 2025
Merged
fix: use proper default and fallback LP solver in scheduler settings (thanks @raffienficiaud)#3736johanneskoester merged 4 commits intomainfrom
johanneskoester merged 4 commits intomainfrom
Conversation
Contributor
📝 WalkthroughWalkthroughAdds get_lp_solvers() to discover available pulp LP solvers (with a fallback), replaces inline pulp probing with that helper, and sets SchedulerSettings.solver default to the first entry of the discovered solver list instead of the fixed literal "COIN_CMD". Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant App as Snakemake (import/init)
participant MILP as milp.get_lp_solvers()
participant Pulp as pulp (optional)
Note over App,MILP: Determine available LP solvers at module init
App->>MILP: call get_lp_solvers()
MILP-->>Pulp: try import pulp & call listSolvers(onlyAvailable=True)
alt pulp available & list succeeds
Pulp-->>MILP: return available solvers
MILP-->>App: return ["PULP_CBC_CMD", ...sorted available solvers...]
else pulp missing or error
MILP-->>App: return ["PULP_CBC_CMD"] (fallback)
end
Note right of App: SchedulerSettings.solver default set to lp_solvers[0]
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Pre-merge checks and finishing touches❌ Failed checks (1 warning, 2 inconclusive)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (1)
🧰 Additional context used📓 Path-based instructions (1)**/*.py⚙️ CodeRabbit configuration file
Files:
🧠 Learnings (2)📓 Common learnings📚 Learning: 2025-07-29T14:53:04.598ZApplied to files:
🧬 Code graph analysis (1)src/snakemake/scheduling/milp.py (1)
🪛 Ruff (0.12.2)src/snakemake/scheduling/milp.py17-21: Consider iterable unpacking instead of concatenation Replace with iterable unpacking (RUF005) 22-22: Do not catch blind exception: (BLE001) ⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (48)
🔇 Additional comments (2)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
johanneskoester
changed the title
johanneskoester
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/snakemake/scheduling/milp.py (1)
52-55: Gracefully handle missing PuLP at runtime to avoid hard crash.If PuLP isn’t installed, the import here raises ImportError and bypasses your existing greedy fallback. Handle it like PulpSolverError to keep behavior consistent.
Apply:
- import pulp - from pulp import lpSum - from pulp import PulpSolverError + try: + import pulp + from pulp import lpSum + from pulp import PulpSolverError + except ImportError as e: + self._technical_failure = True + self.logger.warning( + "PuLP is required for the MILP scheduler but is not installed; " + "falling back to greedy scheduler. Error: %s", + e, + ) + return None
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
src/snakemake/scheduling/milp.py(1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.py
⚙️ CodeRabbit configuration file
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
Files:
src/snakemake/scheduling/milp.py
🧠 Learnings (1)
📚 Learning: 2025-07-29T14:53:04.598Z
Learnt from: johanneskoester
PR: snakemake/snakemake#3676
File: src/snakemake/cli.py:0-0
Timestamp: 2025-07-29T14:53:04.598Z
Learning: In the Snakemake repository, deprecated scheduler interfaces (like scheduler_ilp_solver, --scheduler-solver-path, --scheduler-greediness) should be kept for backward compatibility during the migration to the plugin-based scheduler system, rather than being removed immediately.
Applied to files:
src/snakemake/scheduling/milp.py
🧬 Code graph analysis (1)
src/snakemake/scheduling/milp.py (1)
src/snakemake/scheduling/greedy.py (1)
SchedulerSettings(10-34)
🪛 Ruff (0.12.2)
src/snakemake/scheduling/milp.py
18-20: try-except-pass detected, consider logging the exception
(S110)
18-18: Do not catch blind exception: Exception
(BLE001)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (55)
- GitHub Check: tests (10, windows-2022, py313)
- GitHub Check: tests (10, ubuntu-latest, py311)
- GitHub Check: tests (10, windows-2022, py312)
- GitHub Check: tests (10, ubuntu-latest, py313)
- GitHub Check: tests (10, ubuntu-latest, py312)
- GitHub Check: tests (9, windows-2022, py313)
- GitHub Check: tests (9, ubuntu-latest, py313)
- GitHub Check: tests (8, ubuntu-latest, py312)
- GitHub Check: tests (9, windows-2022, py312)
- GitHub Check: tests (9, ubuntu-latest, py312)
- GitHub Check: tests (8, windows-2022, py313)
- GitHub Check: tests (5, windows-2022, py312)
- GitHub Check: tests (8, ubuntu-latest, py313)
- GitHub Check: tests (8, ubuntu-latest, py311)
- GitHub Check: tests (7, windows-2022, py313)
- GitHub Check: tests (9, ubuntu-latest, py311)
- GitHub Check: tests (8, windows-2022, py312)
- GitHub Check: tests (6, ubuntu-latest, py312)
- GitHub Check: tests (7, ubuntu-latest, py313)
- GitHub Check: tests (7, windows-2022, py312)
- GitHub Check: tests (7, ubuntu-latest, py311)
- GitHub Check: tests (3, windows-2022, py313)
- GitHub Check: tests (6, windows-2022, py312)
- GitHub Check: tests (6, windows-2022, py313)
- GitHub Check: tests (5, ubuntu-latest, py311)
- GitHub Check: tests (7, ubuntu-latest, py312)
- GitHub Check: tests (6, ubuntu-latest, py311)
- GitHub Check: tests (6, ubuntu-latest, py313)
- GitHub Check: tests (3, windows-2022, py312)
- GitHub Check: tests (5, windows-2022, py313)
- GitHub Check: tests (4, ubuntu-latest, py311)
- GitHub Check: tests (2, windows-2022, py313)
- GitHub Check: tests (5, ubuntu-latest, py312)
- GitHub Check: tests (5, ubuntu-latest, py313)
- GitHub Check: tests (4, ubuntu-latest, py312)
- GitHub Check: tests (4, macos-latest, py312)
- GitHub Check: tests (4, ubuntu-latest, py313)
- GitHub Check: tests (4, windows-2022, py312)
- GitHub Check: tests (4, macos-latest, py313)
- GitHub Check: tests (4, windows-2022, py313)
- GitHub Check: tests (3, ubuntu-latest, py311)
- GitHub Check: tests (3, ubuntu-latest, py312)
- GitHub Check: tests (2, macos-latest, py313)
- GitHub Check: tests (3, ubuntu-latest, py313)
- GitHub Check: tests (1, macos-latest, py312)
- GitHub Check: tests (1, windows-2022, py312)
- GitHub Check: tests (1, macos-latest, py313)
- GitHub Check: tests (1, windows-2022, py313)
- GitHub Check: tests (2, windows-2022, py312)
- GitHub Check: tests (2, ubuntu-latest, py312)
- GitHub Check: tests (1, ubuntu-latest, py311)
- GitHub Check: tests (2, ubuntu-latest, py313)
- GitHub Check: tests (1, ubuntu-latest, py313)
- GitHub Check: tests (2, ubuntu-latest, py311)
- GitHub Check: tests (1, ubuntu-latest, py312)
🔇 Additional comments (2)
src/snakemake/scheduling/milp.py (2)
12-13: Good fallback: prefer CBC shipped with PuLP.Using "PULP_CBC_CMD" as the hard fallback keeps lp_solvers non-empty and aligns with typical PuLP installations. LGTM.
26-26: Dynamic default looks correct.Given the non-empty fallback, lp_solvers[0] is safe and matches the PR goal of preferring CBC. LGTM.
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (2)
src/snakemake/scheduling/milp.py (2)
12-21: Make default selection dynamic (prefer CBC if available; otherwise use first available solver deterministically).Right now CBC is always first even when unavailable, which re-triggers the original failure mode; prefer CBC only when present, else use the available set.
Apply:
-def get_lp_solvers(): - default = ["PULP_CBC_CMD"] - try: - import pulp - - return default + sorted( - solver - for solver in pulp.listSolvers(onlyAvailable=True) - if solver != default - ) - except Exception: - return default +def get_lp_solvers(): + preferred = "PULP_CBC_CMD" + fallback = [preferred] + try: + import pulp + except ImportError as e: + logger = logging.getLogger(__name__) + logger.debug("PuLP not available (%s); using fallback solvers: %s", e, fallback) + return fallback + try: + solvers = pulp.listSolvers(onlyAvailable=True) or [] + except Exception as e: + logger = logging.getLogger(__name__) + logger.debug("pulp.listSolvers failed (%s); using fallback solvers: %s", e, fallback) + return fallback + solvers = sorted(set(solvers)) + if preferred in solvers: + return [preferred] + [s for s in solvers if s != preferred] + return solvers or fallbackAdd at module top:
from dataclasses import dataclass, field +import logging import math
22-23: Avoid blind except; catch ImportError and log at debug.Catching all exceptions hides real runtime issues unrelated to import failures.
Apply:
- except Exception: - return default + except ImportError as e: + logger = logging.getLogger(__name__) + logger.debug("PuLP not available (%s); using fallback solvers: %s", e, default) + return defaultNote: If you keep a second try/except around listSolvers, log and fall back there as well.
🧹 Nitpick comments (2)
src/snakemake/scheduling/milp.py (2)
26-26: Double-check import-time detection vs. solver_path influence.lp_solvers is resolved at import time, but solver_path is only applied at solve time. If availability depends on solver_path, choices/default may not reflect reality. Consider late-binding default (e.g., set default=None and decide in post_init) or not filtering by availability for choices.
31-36: Metadata choices should not contain duplicates.Once the filter bug is fixed, this is fine. If you keep set()-based de-dupe in get_lp_solvers, choices will be clean and deterministic.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
src/snakemake/scheduling/milp.py(1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.py
⚙️ CodeRabbit configuration file
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
Files:
src/snakemake/scheduling/milp.py
🧠 Learnings (2)
📓 Common learnings
Learnt from: johanneskoester
PR: snakemake/snakemake#3676
File: src/snakemake/cli.py:0-0
Timestamp: 2025-07-29T14:53:04.598Z
Learning: In the Snakemake repository, deprecated scheduler interfaces (like scheduler_ilp_solver, --scheduler-solver-path, --scheduler-greediness) should be kept for backward compatibility during the migration to the plugin-based scheduler system, rather than being removed immediately.
📚 Learning: 2025-07-29T14:53:04.598Z
Learnt from: johanneskoester
PR: snakemake/snakemake#3676
File: src/snakemake/cli.py:0-0
Timestamp: 2025-07-29T14:53:04.598Z
Learning: In the Snakemake repository, deprecated scheduler interfaces (like scheduler_ilp_solver, --scheduler-solver-path, --scheduler-greediness) should be kept for backward compatibility during the migration to the plugin-based scheduler system, rather than being removed immediately.
Applied to files:
src/snakemake/scheduling/milp.py
🧬 Code graph analysis (1)
src/snakemake/scheduling/milp.py (1)
src/snakemake/scheduling/greedy.py (1)
SchedulerSettings(10-34)
🪛 Ruff (0.12.2)
src/snakemake/scheduling/milp.py
22-22: Do not catch blind exception: Exception
(BLE001)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (55)
- GitHub Check: tests (10, windows-2022, py313)
- GitHub Check: tests (9, windows-2022, py313)
- GitHub Check: tests (10, ubuntu-latest, py312)
- GitHub Check: tests (10, macos-latest, py313)
- GitHub Check: tests (10, ubuntu-latest, py311)
- GitHub Check: tests (10, ubuntu-latest, py313)
- GitHub Check: tests (10, windows-2022, py312)
- GitHub Check: tests (9, windows-2022, py312)
- GitHub Check: tests (9, ubuntu-latest, py313)
- GitHub Check: tests (8, ubuntu-latest, py312)
- GitHub Check: tests (9, ubuntu-latest, py311)
- GitHub Check: tests (9, ubuntu-latest, py312)
- GitHub Check: tests (8, windows-2022, py313)
- GitHub Check: tests (8, windows-2022, py312)
- GitHub Check: tests (8, ubuntu-latest, py313)
- GitHub Check: tests (6, macos-latest, py312)
- GitHub Check: tests (7, ubuntu-latest, py311)
- GitHub Check: tests (7, windows-2022, py312)
- GitHub Check: tests (6, ubuntu-latest, py312)
- GitHub Check: tests (8, ubuntu-latest, py311)
- GitHub Check: tests (5, windows-2022, py313)
- GitHub Check: tests (6, ubuntu-latest, py313)
- GitHub Check: tests (6, windows-2022, py312)
- GitHub Check: tests (7, windows-2022, py313)
- GitHub Check: tests (7, ubuntu-latest, py312)
- GitHub Check: tests (7, ubuntu-latest, py313)
- GitHub Check: tests (6, windows-2022, py313)
- GitHub Check: tests (6, ubuntu-latest, py311)
- GitHub Check: tests (5, windows-2022, py312)
- GitHub Check: tests (5, ubuntu-latest, py312)
- GitHub Check: tests (5, ubuntu-latest, py313)
- GitHub Check: tests (5, ubuntu-latest, py311)
- GitHub Check: tests (4, windows-2022, py312)
- GitHub Check: tests (4, ubuntu-latest, py313)
- GitHub Check: tests (2, windows-2022, py313)
- GitHub Check: tests (4, windows-2022, py313)
- GitHub Check: tests (4, ubuntu-latest, py312)
- GitHub Check: tests (3, windows-2022, py313)
- GitHub Check: tests (4, ubuntu-latest, py311)
- GitHub Check: tests (3, macos-latest, py313)
- GitHub Check: tests (3, ubuntu-latest, py313)
- GitHub Check: tests (3, windows-2022, py312)
- GitHub Check: tests (3, ubuntu-latest, py312)
- GitHub Check: tests (3, ubuntu-latest, py311)
- GitHub Check: tests (2, ubuntu-latest, py312)
- GitHub Check: tests (2, windows-2022, py312)
- GitHub Check: tests (1, macos-latest, py313)
- GitHub Check: tests (1, ubuntu-latest, py313)
- GitHub Check: tests (1, ubuntu-latest, py312)
- GitHub Check: tests (1, windows-2022, py313)
- GitHub Check: tests (2, ubuntu-latest, py313)
- GitHub Check: tests (1, macos-latest, py312)
- GitHub Check: tests (2, ubuntu-latest, py311)
- GitHub Check: tests (1, windows-2022, py312)
- GitHub Check: tests (1, ubuntu-latest, py311)
johanneskoester
pushed a commit
that referenced
this pull request
Sep 18, 2025
🤖 I have created a release *beep* *boop* --- ## [9.11.3](v9.11.2...v9.11.3) (2025-09-16) ### Bug Fixes * logging refinements ([#3571](#3571)) ([47ae16e](47ae16e)) * use proper default and fallback LP solver in scheduler settings (thanks [@raffienficiaud](https://github.com/raffienficiaud)) ([#3736](#3736)) ([e00e5aa](e00e5aa)) ### Documentation * mention nodefaults for installation ([a6db049](a6db049)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
kjohnsen
pushed a commit
to kjohnsen/snakemake
that referenced
this pull request
Dec 15, 2025
…(thanks @raffienficiaud) (snakemake#3736) fixes snakemake#3719 <!--Add a description of your PR here--> ### QC <!-- Make sure that you can tick the boxes below. --> * [x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Default MILP solver is now chosen automatically from solvers available in your environment. * **Improvements** * Solver selection is more robust by using an ordered list of available solvers instead of a fixed default. * Falls back to a sensible bundled solver when others aren’t available. * Users can still pick a solver explicitly via settings or configuration. * **Bug Fixes** * Reduces failures on systems where the previously assumed default solver was missing. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
kjohnsen
pushed a commit
to kjohnsen/snakemake
that referenced
this pull request
Dec 15, 2025
🤖 I have created a release *beep* *boop* --- ## [9.11.3](snakemake/snakemake@v9.11.2...v9.11.3) (2025-09-16) ### Bug Fixes * logging refinements ([snakemake#3571](snakemake#3571)) ([47ae16e](snakemake@47ae16e)) * use proper default and fallback LP solver in scheduler settings (thanks [@raffienficiaud](https://github.com/raffienficiaud)) ([snakemake#3736](snakemake#3736)) ([e00e5aa](snakemake@e00e5aa)) ### Documentation * mention nodefaults for installation ([a6db049](snakemake@a6db049)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
2 tasks
johanneskoester
added a commit
that referenced
this pull request
Jan 7, 2026
Hi everyone, The enumeration of available solvers with `pulp.listSolvers(onlyAvailable=True)` typically acquires license slots of all installed commercial solvers, which becomes problematic, if only a limited number of shared slots are available. pulp correctly releases the licenses immediately again, but it still repeatedly leads to friction. This PR avoids the pulp solver enumeration by using `pulp.getSolver(...).available()` to check for whether the chosen solver is available and only on failure queries the full list. The default solver choice from #3736 was updated to rely on `pulp.apis.LpSolverDefault`, which is compatible with "PULP_CBC_CMD" (if it is available). I also moved the pulp specific checking out of api.py entirely, so that all the solver choice is now handled by the new structure. With these changes, even if pulp is not installed, snakemake gracefully falls back to the greedy scheduler. ### QC <!-- Make sure that you can tick the boxes below. --> * [x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Improvements** * Lazy, more robust solver discovery with improved default selection for ILP scheduling, reducing startup overhead and surprises. * Scheduler now detects when the configured solver is unavailable and automatically falls back to the greedy scheduler. * **New Features** * Exposes a solver-availability indicator for easier diagnostics and decisions. * **Bug Fixes** * Clearer warning when falling back, indicating missing solver support and required external solver dependencies. <sub>✏️ Tip: You can customize this high-level summary in your review settings.</sub> <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Johannes Köster <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fixes #3719
QC
docs/) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).Summary by CodeRabbit
New Features
Improvements
Bug Fixes