Fix multiprocessing.Manager resource leak in concurrent diff processing#4953
Closed
Bitshifter-9 wants to merge 5 commits intopsf:mainfrom
Closed
Fix multiprocessing.Manager resource leak in concurrent diff processing#4953Bitshifter-9 wants to merge 5 commits intopsf:mainfrom
Bitshifter-9 wants to merge 5 commits intopsf:mainfrom
Conversation
- Remove unused imports (Manager, call) - Add type annotations to all functions - Fix line length violations (break lines > 80 chars) - Fix import order per Black conventions
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes a resource leak where multiprocessing.Manager() instances created for diff output modes (--diff or --color-diff) were never properly shut down, potentially leaving background processes running. The fix adds a try-finally block to ensure manager.shutdown() is called even when exceptions occur during formatting.
Changes:
- Initialize
managervariable toNoneand wrap the main formatting loop in a try-finally block that callsmanager.shutdown()when the manager exists - Add comprehensive test suite to verify Manager cleanup on success, exception, non-diff modes, and early return paths
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| src/black/concurrency.py | Adds manager variable initialization and try-finally block to ensure Manager.shutdown() is called |
| tests/test_manager_cleanup.py | New test suite verifying Manager cleanup in various scenarios |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Author
- Use actual source files ({Path('test.py')}) instead of empty set
- Properly mock executor.submit() to return completed Futures
- Tests now correctly FAIL on base commit (shutdown not called)
- Tests PASS on fixed commit (shutdown called in finally block)
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When Black is run with --diff or --color-diff, it creates a multiprocessing.Manager() to share locks across worker processes. This Manager was never explicitly shut down, which can leave background processes running after Black finishes. This becomes problematic when Black is used as a library or invoked repeatedly in long-running environments such as editor integrations or CI pipelines.
This change ensures deterministic cleanup of the Manager by wrapping the main execution path in schedule_formatting() with a try–finally block and calling manager.shutdown() whenever a Manager is created, even if an exception occurs. The fix is intentionally minimal to avoid unnecessary diff noise and does not alter existing behavior beyond proper resource cleanup.
A new test suite in tests/test_manager_cleanup.py verifies that Manager.shutdown() is called on both successful execution and error paths, that no Manager is created when diff output modes are not used, and that early return paths do not create a Manager. All existing tests continue to pass, confirming backward compatibility and no functional regressions.
This prevents resource leaks and stray processes when Black is used repeatedly with diff output modes.
these fixes #4950
Closes #4952