Skip to content

Fix dtype cast in TimesFM 2.5 ResidualBlock for quantized weights - #46942

Merged
SunMarc merged 3 commits into
huggingface:mainfrom
JOhnsonKC201:fix-timesfm2_5-quantized-dtype-cast
Oct 21, 2026
Merged

SunMarc merged 3 commits into
huggingface:mainfrom
JOhnsonKC201:fix-timesfm2_5-quantized-dtype-cast

Conversation

@JOhnsonKC201

@JOhnsonKC201 JOhnsonKC201 commented Jun 28, 2026 •

Copy link
Copy Markdown
Contributor

🤗 Transformers CI

What does this PR do?

TimesFm2_5ResidualBlock.forward unconditionally casts its float input to the block's weight dtype:

x = x.to(self.input_layer.weight.dtype)

input_layer is an nn.Linear, and this block is used as the model's input projection (input_ff_layer), so the raw float time-series inputs flow through it. When the model is loaded with quantization (e.g. bitsandbytes 4/8-bit), input_layer.weight.dtype is an integer type (e.g. uint8). Casting the float inputs to an int dtype silently truncates them to zeros (0.73 -> 0), so the model produces garbage outputs with no error or warning.

The fix only casts when the weight dtype is floating point, preserving the original mixed-precision behavior while skipping the destructive cast under quantization:

if (target_dtype := self.input_layer.weight.dtype).is_floating_point:
    x = x.to(target_dtype)

This is the same pattern already applied to the multimodal embedders in gemma4_unified (#46904) and gemma4 (#46933).

Both modular_timesfm2_5.py and the generated modeling_timesfm2_5.py are updated to stay in sync.

Before submitting

Who can review?

@kashif @SunMarc

`TimesFm2_5ResidualBlock.forward` casts its (float) input to
`self.input_layer.weight.dtype`. Under bitsandbytes 4/8-bit quantization
that weight dtype is an integer type (e.g. uint8), so the cast silently
truncates the float inputs to zeros (0.73 -> 0), producing garbage
outputs with no error or warning. This block is used as the model's input
projection (`input_ff_layer`), so the raw time-series inputs are affected.

Only cast when the weight dtype is floating point, matching the fix
already applied to gemma4_unified (huggingface#46904) and gemma4 (huggingface#46933).
@JOhnsonKC201

Copy link
Copy Markdown
Contributor Author

Small, self-contained fix (+10/-4, with modular_timesfm2_5.py and the generated modeling_timesfm2_5.py kept in sync). The destructive cast only triggers under quantization, where input_layer.weight.dtype is an integer type — so the float time-series inputs get silently truncated to zeros with no error in normal fp runs. Same failure mode and fix as the multimodal embedders in #46904 and #46933.

Happy to run run-slow: timesfm2_5 or adjust anything once a maintainer has a chance to look. Thanks!

@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@SunMarc SunMarc left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed thanks !

@SunMarc
SunMarc enabled auto-merge September 17, 2026 16:00
@ArthurZucker
ArthurZucker requested a review from vasqu October 21, 2026 01:34
@ArthurZucker

Copy link
Copy Markdown
Collaborator

@vasqu this was approved by @SunMarc on 2026-09-17; could you give it the core-maintainer review so it can be merged?

@github-actions

Copy link
Copy Markdown
Contributor

Thank you for your contribution 🤗!

CI Security Gate — waiting for maintainer approval

CI is approved automatically only for authors who already have a contribution merged in this repository. A maintainer will review this PR and approve CI to run on it.

@github-actions

Copy link
Copy Markdown
Contributor

[For maintainers] Suggested jobs to run (before merge)

run-slow: timesfm2_5

@vasqu

vasqu commented Oct 21, 2026

Copy link
Copy Markdown
Collaborator

run-slow: timesfm2_5

@SunMarc
SunMarc added this pull request to the merge queue Oct 21, 2026
Merged via the queue into huggingface:main with commit 0d65c9c Oct 21, 2026
41 of 42 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

CI (Nvidia + AMD)

Nvidia · ✅ No new failures 🎉👏 · ⚙️ Workflow · 📊 Dashboard

AMD · ✅ No new failures 🎉👏 · ⚙️ Workflow · 📊 Dashboard

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants