Skip to content

Remove invalid @staticmethod from module-level get_device_and_memory_breakdown #41747

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
albertvillanova merged 1 commit into from
Oct 22, 2025
Merged

Remove invalid @staticmethod from module-level get_device_and_memory_breakdown #41747

albertvillanova merged 1 commit into from
Oct 22, 2025

Conversation

@albertvillanova
Copy link
Member

@albertvillanova albertvillanova commented Oct 20, 2025 •
edited
Loading

Remove invalid staticmethod decorator from module-level function get_device_and_memory_breakdown.

This PR removes an incorrect @staticmethod decorator applied to the module-level function get_device_and_memory_breakdown.

The decorator caused a runtime error on Python 3.9, even though it appeared to work fine on newer Python versions.

See trl stacktrace for Python 3.9: https://github.com/huggingface/trl/actions/runs/18605452485/job/53053774127

ERROR    ContinuousBatchingLogger:continuous_api.py:879 Error in generation loop: 'staticmethod' object is not callable
Traceback (most recent call last):
  File "/__w/trl/trl/.venv/lib/python3.9/site-packages/transformers/generation/continuous_batching/continuous_api.py", line 837, in _run_generation_loop
    paged_attention_cache = PagedAttentionCache(
  File "/__w/trl/trl/.venv/lib/python3.9/site-packages/transformers/generation/continuous_batching/cache.py", line 191, in __init__
    num_blocks, max_batch_tokens = memory_handler.infer_num_blocks_and_max_batch_tokens(
  File "/__w/trl/trl/.venv/lib/python3.9/site-packages/transformers/generation/continuous_batching/cache.py", line 437, in infer_num_blocks_and_max_batch_tokens
    num_blocks, max_batch_tokens = self.compute_num_blocks_and_max_batch_tokens(
  File "/__w/trl/trl/.venv/lib/python3.9/site-packages/transformers/generation/continuous_batching/cache.py", line 476, in compute_num_blocks_and_max_batch_tokens
    cache_memory = self.get_available_memory(max_memory_percent)
  File "/__w/trl/trl/.venv/lib/python3.9/site-packages/transformers/generation/continuous_batching/cache.py", line 411, in get_available_memory
    _, total, reserved, allocated = get_device_and_memory_breakdown()
TypeError: 'staticmethod' object is not callable

Root cause

In Python 3.9 and earlier, @staticmethod produces a descriptor object that is not directly callable when defined outside a class.

Starting with Python 3.10, CPython changed the behavior of staticmethod: staticmethod objects gained a __call__ method that delegates to the wrapped function, making them callable even outside a class context. However, this masked the underlying issue in newer Python versions.

Fix

Remove the invalid decorator and leave the function as a normal callable at module scope.

This PR will fix huggingface/trl#4308

CC: @remi-or, who created the original PR:

@albertvillanova albertvillanova mentioned this pull request Oct 20, 2025
Closed
@albertvillanova
Copy link
Member Author

albertvillanova commented Oct 20, 2025 •
edited
Loading

If approved, this fix may need being backported to v4.57-release branch, which still supports Python 3.9.

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Oct 20, 2025

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

remi-or
remi-or approved these changes Oct 20, 2025
View reviewed changes
Copy link
Collaborator

@remi-or remi-or left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing! This was quite ugly.

@albertvillanova albertvillanova merged commit 4f8781f into huggingface:main Oct 22, 2025
23 checks passed
@vasqu vasqu added the for patch Tag issues / labels that should be included in the next patch label Oct 22, 2025
@vasqu
Copy link
Contributor

vasqu commented Oct 22, 2025

cc @ArthurZucker @Cyrilvallez in case we have another patch for v4, I added a for patch label

@albertvillanova
Copy link
Member Author

albertvillanova commented Oct 22, 2025

Thanks, @vasqu. As I commented above: #41747 (comment)

If approved, this fix may need being backported to v4.57-release branch, which still supports Python 3.9.

ngazagna-qc pushed a commit to ngazagna-qc/transformers that referenced this pull request Oct 23, 2025
@albertvillanova @ngazagna-qc
Remove invalid @staticmethod from module-level get_device_and_memor…
2725f83 
…y_breakdown (huggingface#41747)

Remove staticmethod decorator from function
@albertvillanova albertvillanova mentioned this pull request Oct 30, 2025
Merged
i3hz pushed a commit to i3hz/transformers that referenced this pull request Oct 30, 2025
@albertvillanova @i3hz
Remove invalid @staticmethod from module-level get_device_and_memor…
8d67d36 
…y_breakdown (huggingface#41747)

Remove staticmethod decorator from function
Cyrilvallez pushed a commit that referenced this pull request Nov 24, 2025
@albertvillanova @Cyrilvallez
Remove invalid @staticmethod from module-level get_device_and_memor…
311807f 
…y_breakdown (#41747)

Remove staticmethod decorator from function
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@remi-or remi-or remi-or approved these changes

Assignees

No one assigned

Labels

for patch Tag issues / labels that should be included in the next patch

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

CI fails for Python 3.9: ValueError: zero-size array to reduction operation maximum which has no identity

4 participants

@albertvillanova @HuggingFaceDocBuilderDev @vasqu @remi-or