Skip to content

fix(cmmmu): normalize image media inputs - #1618

Merged
Yunnglin merged 1 commit into
modelscope:mainfrom
Excelius-Wang:fix/cmmmu-media-loading
Aug 24, 2026
Merged

Yunnglin merged 1 commit into
modelscope:mainfrom
Excelius-Wang:fix/cmmmu-media-loading

Conversation

@Excelius-Wang

Copy link
Copy Markdown
Contributor

Summary

  • migrate CMMMU image loading to the shared _extract_media path
  • support path-backed Hugging Face image dictionaries and detect image MIME types from bytes
  • bump the CMMMU evaluation version to v1.1
  • add regression coverage for path-backed and ordered multi-image inputs

Problem

The CMMMU adapter reads image_<n>["bytes"] directly and labels every image data URI as PNG. A valid undecoded image value with bytes=None and a usable path therefore raises TypeError, while JPEG bytes are sent with an incorrect PNG MIME type.

Validation

  • pytest -q tests/benchmark/test_cmmmu_adapter.py tests/api/test_vision_language_adapter.py tests/benchmark/test_general_vmcq_adapter.py — 25 passed
  • pytest tests/cli/test_all.py::TestRun::test_ci_lite -q -p no:warnings — 1 passed
  • pre-commit hooks passed for both changed files
  • checked 100 records and 111 images from the official CMMMU validation Parquet: content shape, text, image order, and Base64 payloads matched the legacy path; 43 incorrect PNG MIME headers were corrected

Part of #1605.

@Yunnglin
Yunnglin merged commit e602098 into modelscope:main Aug 24, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants