Skip to content

fix failed test cases for qwen3_omni_moe model#47449

Merged
zucchini-nlp merged 5 commits into
huggingface:mainfrom
kaixuanliu:qwen3_omni_moe_fix
Jul 22, 2026
Merged

fix failed test cases for qwen3_omni_moe model#47449
zucchini-nlp merged 5 commits into
huggingface:mainfrom
kaixuanliu:qwen3_omni_moe_fix

Conversation

@kaixuanliu

@kaixuanliu kaixuanliu commented Jul 21, 2026 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Contributor

CI

This PR tries to fix following failed test cases:

tests/models/qwen3_omni_moe/test_modeling_qwen3_omni_moe.py::Qwen3OmniModelIntegrationTest::test_small_model_integration_test_batch_flashatt2

tests/models/qwen3_omni_moe/test_modeling_qwen3_omni_moe.py::Qwen3OmniMoeThinkerForConditionalGenerationModelTest::test_flash_attention_2_padding_matches_padding_free_with_position_ids

tests/models/qwen3_omni_moe/test_modeling_qwen3_omni_moe.py::Qwen3OmniMoeThinkerForConditionalGenerationModelTest::test_flash_attention_2_padding_matches_padding_free_with_position_ids_and_fa_kwargs

tests/models/qwen3_omni_moe/test_modeling_qwen3_omni_moe.py::Qwen3OmniMoeThinkerForConditionalGenerationModelTest::test_generate_with_quant_cache

tests/models/qwen3_omni_moe/test_modeling_qwen3_omni_moe.py::Qwen3OmniMoeThinkerForConditionalGenerationModelTest::test_model_parallelism

tests/models/qwen3_omni_moe/test_modeling_qwen3_omni_moe.py::Qwen3OmniModelIntegrationTest::test_small_model_integration_test

tests/models/qwen3_omni_moe/test_modeling_qwen3_omni_moe.py::Qwen3OmniModelIntegrationTest::test_small_model_integration_test_batch

tests/models/qwen3_omni_moe/test_modeling_qwen3_omni_moe.py::Qwen3OmniModelIntegrationTest::test_small_model_integration_test_multiturn

tests/models/qwen3_omni_moe/test_modeling_qwen3_omni_moe.py::Qwen3OmniModelIntegrationTest::test_small_model_integration_test_w_audio

@zucchini-nlp @ydshieh pls help review, thx!

@kaixuanliu
kaixuanliu marked this pull request as ready for review July 21, 2026 09:20
@kaixuanliu
FIX LINT issue
1f13338 
Signed-off-by: kaixuanliu <[email protected]>
zucchini-nlp
zucchini-nlp reviewed Jul 21, 2026
View reviewed changes
Comment thread src/transformers/models/qwen3_omni_moe/modular_qwen3_omni_moe.py
Comment on lines +684 to +688
_no_split_modules = [
"Qwen3OmniMoeThinkerTextDecoderLayer",
"Qwen3OmniMoeVisionBlock",
]

@zucchini-nlp zucchini-nlp Jul 21, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you please get all no-split modules and define them once in here? When loading a backbone, we'll filter all available modules from the list anyway and it makes it easy if we keep one list

I am seeing that vision model override it and define ["Qwen3OmniMoeVisionEncoder"], then audio overrides as ["Qwen3OmniMoeAudioEncoder"] atm

@kaixuanliu kaixuanliu Jul 22, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, done for it.

@kaixuanliu
update code
4d9cfc6 
Signed-off-by: kaixuanliu <[email protected]>
@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

[For maintainers] Suggested jobs to run (before merge)

run-slow: qwen3_omni_moe

@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

CI recap

Dashboard: View test results in Grafana
Latest run: 29887252629:2
Result: success | Jobs: 2 | Tests: 24 | Failures: 0 | Duration: 2m 16s

zucchini-nlp
zucchini-nlp approved these changes Jul 22, 2026
View reviewed changes

@zucchini-nlp zucchini-nlp left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@zucchini-nlp
zucchini-nlp enabled auto-merge July 22, 2026 07:51
@zucchini-nlp
zucchini-nlp added this pull request to the merge queue Jul 22, 2026
Merged via the queue into huggingface:main with commit 9f5638e Jul 22, 2026
36 checks passed
@HuggingFaceDocBuilderDev

HuggingFaceDocBuilderDev commented Jul 22, 2026

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@kaixuanliu
kaixuanliu deleted the qwen3_omni_moe_fix branch July 22, 2026 12:02
SangbumChoi added a commit to SangbumChoi/transformers that referenced this pull request Jul 22, 2026
@SangbumChoi
Merge remote-tracking branch 'upstream/main' into locateanything
3209b96 
* upstream/main: (39 commits)
  Remove deprecated training args and `is_fast` property (huggingface#46917)
  Consistent output shape from `get_image_features` (huggingface#46405)
  Fix multi-device mxfp4 dequantization race in `_convert_moe_packed_tensors` (huggingface#47423)
  fix failed test cases for qwen3_omni_moe model (huggingface#47449)
  Fix Hunyuan-VL PIL image resize parity with reference preprocessing (huggingface#47233)
  Move `value` padding into the attention interfaces that need it (huggingface#47451)
  Simplify function dispatch for linear attention (huggingface#47450)
  [cache] Allow sliding window layers to be roll-backed for speculative decoding (huggingface#47447)
  Fix double-shifted training loss in GitForCausalLM (huggingface#47395)
  Fix CohereASR training-loss double-shift (same as Moonshine fix huggingface#46784) (huggingface#46895)
  Warn when `group_by_length` is silently ignored for iterable datasets (huggingface#47379)
  Update bug report list (huggingface#46607)
  Fix shape mismatch in KyutaiSpeechToText `generate()` last window (huggingface#46952)
  Optimize flash attention max seqlen computation in vision attention (huggingface#47170)
  fix: remove unreachable return in special token builder (huggingface#47420)
  Add Harry to slow CI (huggingface#47454)
  BLT: vectorize patch length processing (huggingface#47385)
  Fix `TrackioCallback` fails to log evaluation metrics after training ends (huggingface#46935)
  [Kimi] add integration tests (huggingface#47383)
  Fix typo in `MusicgenForCausalLM.generate()` (huggingface#46974)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zucchini-nlp zucchini-nlp zucchini-nlp approved these changes
@Rocketknight1 Rocketknight1 Awaiting requested review from Rocketknight1
@ydshieh ydshieh Awaiting requested review from ydshieh

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@kaixuanliu @HuggingFaceDocBuilderDev @zucchini-nlp