System Info
N/A.
Who can help?
The hard coded numbers 12 and -1 seem to be related to attention_context_left and attention_context_right.
|
position_ids = torch.arange(12, -1, -1, device=hidden_states.device) |
@eustlb @ebezzam @vasqu
Information
Tasks
Reproduction
Code review.
Expected behavior
Use params from config.
System Info
N/A.
Who can help?
The hard coded numbers 12 and -1 seem to be related to
attention_context_leftandattention_context_right.transformers/src/transformers/models/gemma4/modular_gemma4.py
Line 160 in 8426e7e
@eustlb @ebezzam @vasqu
Information
Tasks
examplesfolder (such as GLUE/SQuAD, ...)Reproduction
Code review.
Expected behavior
Use params from
config.