fix: shorten report ids (thus dir names) in order to avoid issues with path length on windows#3822
Merged
johanneskoester merged 4 commits intomainfrom Nov 4, 2025
Merged
Conversation
Contributor
📝 WalkthroughWalkthroughAdded a new public function Changes
Sequence Diagram(s)sequenceDiagram
%% Styling: green notes highlight new/changed interactions
participant Expander as expand_cat_name flow
participant Shortener as shorten_ids
participant Assembler as assemble results (auto_report)
participant Renderer as sort & render
Expander->>Shortener: pass results (post-expand)
Shortener-->>Expander: mutate ids in-place
Assembler->>Shortener: pass assembled results
Shortener-->>Assembler: mutate ids in-place
Assembler->>Renderer: proceed to sort & render
note right of Shortener #DDFFDD: Ensures IDs unique (8–64 chars) or logs warning
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes
Pre-merge checks and finishing touches❌ Failed checks (2 inconclusive)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (46)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
johanneskoester
changed the title
Contributor
|
Please format your code with |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
src/snakemake/report/__init__.py (1)
458-460: Add return type annotation.Since this function contains a return statement and is a public function, adding a return type annotation would improve clarity.
Apply this diff:
-def shorten_ids(results: Mapping[ - Category, Mapping[Category, List[FileRecord]] -]): +def shorten_ids(results: Mapping[ + Category, Mapping[Category, List[FileRecord]] +]) -> None:
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
src/snakemake/report/__init__.py(3 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.py
⚙️ CodeRabbit configuration file
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
Files:
src/snakemake/report/__init__.py
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (45)
- GitHub Check: tests (9, windows-2022, py313)
- GitHub Check: tests (10, windows-2022, py313)
- GitHub Check: tests (7, ubuntu-latest, py313)
- GitHub Check: tests (10, ubuntu-latest, py313)
- GitHub Check: tests (6, ubuntu-latest, py313)
- GitHub Check: tests (9, ubuntu-latest, py311)
- GitHub Check: tests (7, ubuntu-latest, py312)
- GitHub Check: tests (10, ubuntu-latest, py312)
- GitHub Check: tests (9, ubuntu-latest, py313)
- GitHub Check: tests (8, windows-2022, py313)
- GitHub Check: tests (9, ubuntu-latest, py312)
- GitHub Check: tests (10, ubuntu-latest, py311)
- GitHub Check: tests (8, ubuntu-latest, py312)
- GitHub Check: tests (7, ubuntu-latest, py311)
- GitHub Check: tests (8, ubuntu-latest, py311)
- GitHub Check: tests (1, windows-2022, py313)
- GitHub Check: tests (8, ubuntu-latest, py313)
- GitHub Check: tests (7, windows-2022, py313)
- GitHub Check: tests (6, ubuntu-latest, py312)
- GitHub Check: tests (6, ubuntu-latest, py311)
- GitHub Check: tests (6, windows-2022, py313)
- GitHub Check: tests (5, ubuntu-latest, py313)
- GitHub Check: tests (5, ubuntu-latest, py311)
- GitHub Check: tests (5, macos-latest, py313)
- GitHub Check: tests (4, windows-2022, py313)
- GitHub Check: tests (5, windows-2022, py313)
- GitHub Check: tests (4, macos-latest, py313)
- GitHub Check: tests (4, ubuntu-latest, py313)
- GitHub Check: tests (5, ubuntu-latest, py312)
- GitHub Check: tests (4, ubuntu-latest, py312)
- GitHub Check: tests (3, ubuntu-latest, py313)
- GitHub Check: tests (1, ubuntu-latest, py313)
- GitHub Check: tests (2, ubuntu-latest, py313)
- GitHub Check: tests (3, ubuntu-latest, py311)
- GitHub Check: tests (2, macos-latest, py313)
- GitHub Check: tests (4, ubuntu-latest, py311)
- GitHub Check: tests (3, windows-2022, py313)
- GitHub Check: tests (1, ubuntu-latest, py311)
- GitHub Check: tests (3, ubuntu-latest, py312)
- GitHub Check: tests (3, macos-latest, py313)
- GitHub Check: tests (2, ubuntu-latest, py311)
- GitHub Check: tests (2, ubuntu-latest, py312)
- GitHub Check: tests (2, windows-2022, py313)
- GitHub Check: tests (1, macos-latest, py313)
- GitHub Check: tests (1, ubuntu-latest, py312)
🔇 Additional comments (3)
src/snakemake/report/__init__.py (3)
15-15: LGTM! Import addition is appropriate.The
Mappingimport is correctly added and used in the type annotation of theshorten_idsfunction.
676-676: Good placement of the shorten_ids call.Calling
shorten_ids(results)here is appropriate—after all FileRecords have been collected but before final sorting and rendering. This ensures all IDs are available for uniqueness checking.
676-676: AI summary inconsistency: only one call to shorten_ids.The AI summary states that
shorten_idsis called in two places, including "After expand_cat_name within expand_labels/expand_cat_name flow." However, the code shows only this single call at line 676. The single call is correctly placed, but the summary is inaccurate.
Comment on lines
+458
to
+472
| def shorten_ids(results: Mapping[ | ||
| Category, Mapping[Category, List[FileRecord]] | ||
| ]): | ||
| file_records = [ | ||
| res | ||
| for cat, subcats in results.items() | ||
| for subcat, catresults in subcats.items() | ||
| for res in catresults | ||
| ] | ||
| for id_len in range(8, 65): | ||
| if len({rec.id[:id_len] for rec in file_records}) == len(file_records): | ||
| for rec in file_records: | ||
| rec.id = rec.id[:id_len] | ||
| return | ||
|
|
Contributor
There was a problem hiding this comment.
Handle the case when uniqueness cannot be achieved.
If no length from 8 to 64 produces unique IDs, the function exits silently without shortening the IDs or logging a warning. This could leave long IDs that cause Windows path length issues—the very problem this PR aims to solve.
Consider one of these approaches:
- Log a warning and keep the full-length IDs if uniqueness cannot be achieved:
def shorten_ids(results: Mapping[
Category, Mapping[Category, List[FileRecord]]
]):
file_records = [
res
for cat, subcats in results.items()
for subcat, catresults in subcats.items()
for res in catresults
]
for id_len in range(8, 65):
if len({rec.id[:id_len] for rec in file_records}) == len(file_records):
for rec in file_records:
rec.id = rec.id[:id_len]
return
+ logger.warning(
+ "Could not shorten report IDs while maintaining uniqueness. "
+ "This may cause path length issues on Windows."
+ )- Or, truncate to 64 characters anyway and log a warning about potential ID collisions:
def shorten_ids(results: Mapping[
Category, Mapping[Category, List[FileRecord]]
]):
file_records = [
res
for cat, subcats in results.items()
for subcat, catresults in subcats.items()
for res in catresults
]
for id_len in range(8, 65):
if len({rec.id[:id_len] for rec in file_records}) == len(file_records):
for rec in file_records:
rec.id = rec.id[:id_len]
return
+ # Could not find a unique length; truncate to 64 anyway
+ logger.warning(
+ "Could not shorten report IDs to a unique length. "
+ "Truncating to 64 characters; ID collisions may occur."
+ )
+ for rec in file_records:
+ rec.id = rec.id[:64]📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| def shorten_ids(results: Mapping[ | |
| Category, Mapping[Category, List[FileRecord]] | |
| ]): | |
| file_records = [ | |
| res | |
| for cat, subcats in results.items() | |
| for subcat, catresults in subcats.items() | |
| for res in catresults | |
| ] | |
| for id_len in range(8, 65): | |
| if len({rec.id[:id_len] for rec in file_records}) == len(file_records): | |
| for rec in file_records: | |
| rec.id = rec.id[:id_len] | |
| return | |
| def shorten_ids(results: Mapping[ | |
| Category, Mapping[Category, List[FileRecord]] | |
| ]): | |
| file_records = [ | |
| res | |
| for cat, subcats in results.items() | |
| for subcat, catresults in subcats.items() | |
| for res in catresults | |
| ] | |
| for id_len in range(8, 65): | |
| if len({rec.id[:id_len] for rec in file_records}) == len(file_records): | |
| for rec in file_records: | |
| rec.id = rec.id[:id_len] | |
| return | |
| # Could not find a unique length; truncate to 64 anyway | |
| logger.warning( | |
| "Could not shorten report IDs to a unique length. " | |
| "Truncating to 64 characters; ID collisions may occur." | |
| ) | |
| for rec in file_records: | |
| rec.id = rec.id[:64] |
Log a warning when result IDs are non-unique.
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
src/snakemake/report/__init__.py(3 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.py
⚙️ CodeRabbit configuration file
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
Files:
src/snakemake/report/__init__.py
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (45)
- GitHub Check: tests (10, ubuntu-latest, py313)
- GitHub Check: tests (10, windows-2022, py313)
- GitHub Check: tests (10, ubuntu-latest, py312)
- GitHub Check: tests (9, windows-2022, py313)
- GitHub Check: tests (10, ubuntu-latest, py311)
- GitHub Check: tests (9, ubuntu-latest, py313)
- GitHub Check: tests (9, ubuntu-latest, py312)
- GitHub Check: tests (7, macos-latest, py313)
- GitHub Check: tests (6, ubuntu-latest, py313)
- GitHub Check: tests (7, ubuntu-latest, py313)
- GitHub Check: tests (8, ubuntu-latest, py312)
- GitHub Check: tests (7, windows-2022, py313)
- GitHub Check: tests (9, ubuntu-latest, py311)
- GitHub Check: tests (8, windows-2022, py313)
- GitHub Check: tests (8, ubuntu-latest, py311)
- GitHub Check: tests (5, macos-latest, py313)
- GitHub Check: tests (8, ubuntu-latest, py313)
- GitHub Check: tests (6, ubuntu-latest, py311)
- GitHub Check: tests (5, ubuntu-latest, py313)
- GitHub Check: tests (3, macos-latest, py313)
- GitHub Check: tests (5, ubuntu-latest, py311)
- GitHub Check: tests (5, windows-2022, py313)
- GitHub Check: tests (7, ubuntu-latest, py312)
- GitHub Check: tests (7, ubuntu-latest, py311)
- GitHub Check: tests (6, windows-2022, py313)
- GitHub Check: tests (6, ubuntu-latest, py312)
- GitHub Check: tests (5, ubuntu-latest, py312)
- GitHub Check: tests (3, ubuntu-latest, py313)
- GitHub Check: tests (3, ubuntu-latest, py311)
- GitHub Check: tests (4, windows-2022, py313)
- GitHub Check: tests (3, ubuntu-latest, py312)
- GitHub Check: tests (4, ubuntu-latest, py311)
- GitHub Check: tests (4, ubuntu-latest, py313)
- GitHub Check: tests (4, ubuntu-latest, py312)
- GitHub Check: tests (3, windows-2022, py313)
- GitHub Check: tests (2, macos-latest, py313)
- GitHub Check: tests (1, ubuntu-latest, py311)
- GitHub Check: tests (2, windows-2022, py313)
- GitHub Check: tests (1, ubuntu-latest, py313)
- GitHub Check: tests (2, ubuntu-latest, py312)
- GitHub Check: tests (2, ubuntu-latest, py311)
- GitHub Check: tests (1, macos-latest, py313)
- GitHub Check: tests (2, ubuntu-latest, py313)
- GitHub Check: tests (1, ubuntu-latest, py312)
- GitHub Check: tests (1, windows-2022, py313)
🔇 Additional comments (2)
src/snakemake/report/__init__.py (2)
15-15: LGTM!The
Mappingimport is correctly added to support the type annotation in the newshorten_idsfunction.
675-675: LGTM!The
shorten_idscall is correctly placed after all results are collected and before rendering begins. This ensures all FileRecord IDs are shortened while maintaining uniqueness.
johanneskoester
pushed a commit
that referenced
this pull request
Nov 4, 2025
🤖 I have created a release *beep* *boop* --- ## [9.13.5](v9.13.4...v9.13.5) (2025-11-04) ### Bug Fixes * cache wrapper files and wait for them in case of shared filesystem for sources ([#3809](#3809)) ([498fff7](498fff7)) * correctly handle meta-wrapper tag replacement depending on the used snakemake-wrapper release ([#3826](#3826)) ([8d46a4a](8d46a4a)) * ensure that flags are properly considered for input files before applying path modifiers (i.e. default storage providers) ([#3813](#3813)) ([d3bfe32](d3bfe32)) * ensure that tokens are not leaked when paths or uris of source files are logged ([#3821](#3821)) ([a217e50](a217e50)) * print secs as numeric in jsonl benchmark format ([#3814](#3814)) ([395a5e6](395a5e6)) * revert breaking change in 9.11.9 disallowing empty input files even when unused ([#3810](#3810)) ([83c05cc](83c05cc)) * shorten report ids (thus dir names) in order to avoid issues with path length on windows ([#3822](#3822)) ([b24d971](b24d971)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
kjohnsen
pushed a commit
to kjohnsen/snakemake
that referenced
this pull request
Dec 15, 2025
…h path length on windows (snakemake#3822) ### Description <!--Add a description of your PR here--> ### QC <!-- Make sure that you can tick the boxes below. --> * [x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Reports now automatically shorten file identifiers to more readable lengths while preserving uniqueness across all records, improving clarity in generated reports and UI lists. * Shortened IDs are applied early in processing and again just before final report rendering, ensuring consistent, concise identifiers throughout the reporting flow. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
kjohnsen
pushed a commit
to kjohnsen/snakemake
that referenced
this pull request
Dec 15, 2025
🤖 I have created a release *beep* *boop* --- ## [9.13.5](snakemake/snakemake@v9.13.4...v9.13.5) (2025-11-04) ### Bug Fixes * cache wrapper files and wait for them in case of shared filesystem for sources ([snakemake#3809](snakemake#3809)) ([498fff7](snakemake@498fff7)) * correctly handle meta-wrapper tag replacement depending on the used snakemake-wrapper release ([snakemake#3826](snakemake#3826)) ([8d46a4a](snakemake@8d46a4a)) * ensure that flags are properly considered for input files before applying path modifiers (i.e. default storage providers) ([snakemake#3813](snakemake#3813)) ([d3bfe32](snakemake@d3bfe32)) * ensure that tokens are not leaked when paths or uris of source files are logged ([snakemake#3821](snakemake#3821)) ([a217e50](snakemake@a217e50)) * print secs as numeric in jsonl benchmark format ([snakemake#3814](snakemake#3814)) ([395a5e6](snakemake@395a5e6)) * revert breaking change in 9.11.9 disallowing empty input files even when unused ([snakemake#3810](snakemake#3810)) ([83c05cc](snakemake@83c05cc)) * shorten report ids (thus dir names) in order to avoid issues with path length on windows ([snakemake#3822](snakemake#3822)) ([b24d971](snakemake@b24d971)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
QC
docs/) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).Summary by CodeRabbit