Skip to content

Commit 0fb858c

Browse files
titaiwangmsjustinchubyCopilot
authored
Support Gemma 4 12B Unified native multimodal generation (#2685)
## Summary Continue Justin's original #2286 from an upstream development branch, as requested by the repository owner. The original PR was closed on October 7, 2026 in favor of this replacement. - Recognize/register `gemma4_unified` and wire native image/audio preprocessing into multimodal generation. - Preserve padded 6912-dimensional image patches and positions; consume raw 640-sample audio frames and masks, counting each frame as one audio soft token. - Correct INT64-to-INT32 image-position conversion and test native generation/tensor contracts. - Reject unsupported image-position dtypes before processing; accept only INT32 and INT64, with explicit conversion branches. - Match the Unified dummy speech output to the production rank-two `[num_valid_frames, hidden_size]` contract and test its native embedding handoff. - Stabilize the dummy vision reduction: accumulate in DOUBLE, retain FLOAT outputs, and match the NumPy float64-then-float32 oracle without loosening tolerance. - Make the Unified dummy audio decoder explicitly unlimited, consistent with the exporter. - Replace the contributor-fork dependency with the official Microsoft Extensions URL and immutable landed commit; synchronize its component manifest. Current head: `ef8d5a4bccc1865b8b47fc6657024f07f22607d3`. Companion preprocessing: microsoft/onnxruntime-extensions#1132. Companion exporter: onnxruntime/mobius#759. ## Latest scoped validation - Fresh normal CPU dependency build covering the two latest review fixes, without an Extensions source override: downloaded `https://github.com/microsoft/onnxruntime-extensions.git` at its landed squash-merge commit `99fec2db1c0a17de60eb0f1c2353c91f8d5e22ca`, containing the landed trust-store CI fix microsoft/onnxruntime-extensions#1133. - Imported the freshly rebuilt GenAI package and ran standard, pipeline, and Unified Gemma4 Python cases: **40 passed, no skips**. This includes rank-two speech output/embedding handoff, supported INT32/INT64 image positions, and four unsupported-dtype cases (standard/Unified x FLOAT/FLOAT16). The latter construct a valid model first and check that multimodal processor creation rejects the unsupported input dtype. - Ruff and Black checks passed for the modified Python fixture/tests; targeted clang-format checks passed for the C++ dtype guard. Dependency and component-manifest URL/SHA agree exactly. - Earlier rebase onto official `main` at `45e972d2e09ba41ca59b377e0a819e013129135f` preserved the new upstream pipeline fixtures/tests. The PR's existing modality-signal helpers were given distinct names with matching Unified imports; upstream helper ASTs and renamed PR helper ASTs were checked against both original conflict sides. - Companion Extensions full preprocessing suite: **190 passed, 2 disabled**; seven focused LeakSanitizer tests passed with no leak report after fixing partial-batch cleanup. - Companion Mobius exporter suite: **210 passed**. ## Real-model CPU evidence: bounded smoke, not dataset quality A fresh real `google/gemma-4-12B` FP32 export and native CPU E2E run were completed on October 8, 2026 at the earlier GenAI head `976ae3989ad9f13bf623535ef1e12b6584eac329`, against the same official landed Extensions commit `99fec2db1c0a17de60eb0f1c2353c91f8d5e22ca`. The clean exporter checkout was tree-equivalent to the landed onnxruntime/mobius#759. The checkpoint revision was `023679ed352de9bb66cc873c9009ce3482585c08`. | CPU FP32 case | Bounded execution result | |---|---| | Text only | Eight generated tokens; finite raw logits | | Image | Native vision -> embedding -> decoder; eight generated tokens; finite raw logits | | Two-second audio clip | 50 frames retained, all masks and audio_sizes checked; eight generated tokens; finite raw logits | | 31-second synthetic audio with a final marker | All 775 frames and the final marker retained; eight generated tokens; finite raw logits | All active provider-bearing node events in these four cases used `CPUExecutionProvider`. Export/checkpoint/configuration hashes were checked before and after execution. **Search-policy qualification:** when the raw greedy top token was structural end-of-image/end-of-audio ID 258882 or 258883, both IDs were suppressed after checking the raw logits. This occurred in the text, image, and long-audio cases, not the short-audio case. These are therefore **not unmodified-greedy quality results**. Text/image semantics were not scored; the long synthetic clip generated eight spaces and is not ASR-quality evidence. Timings include startup/loading/profiling and are not a performance benchmark. The two subsequent commits correct the rank-two test fixture and reject unsupported position dtypes; they do not change generation/search logic. Their latest-head coverage is the fresh 40-test CPU suite above, not a claim that this four-case real-model run was repeated at `ef8d5a4`. Earlier real audio-boundary smoke also exercised CPU FP32 and CUDA-configured FP16 with a two-second clip and a 31-second synthetic constant/marker clip, retaining 50/775 frames and finite raw logits. Those runs preceded the final Extensions ownership-only cleanup, which is covered separately by LeakSanitizer and full preprocessing tests. CUDA decoder profiles included CUDA and CPU events; child-session CUDA execution was unproven, and disabling CPU fallback rejected construction. Two FP32 decoder constants remained in the FP16 audit. Pure-CUDA execution and an entirely FP16 graph are not claimed. ## Historical dataset integration and quality evidence Full original summary, source revisions, and qualifications: microsoft/onnxruntime-extensions#1091 (comment) These are **historical GPU dataset measurements**, retained as supporting integration evidence, not fresh CPU dataset results or a rerun at the current head. The earlier GenAI head `eacf6b6121528728108346e847be7350d2d3f535` also passed 23 focused Python tests, a fresh dependency build, and clean-Mobius real-model text/image/audio smoke. | Earlier bounded E2B main/main vs PR/PR comparison, same FP16 ONNX package | Result | |---|---| | First 100 AI2D samples | Identical answers/tokens/tensor digests; 58% accuracy both | | First 100 FLEURS samples | Identical transcripts/tokens/tensor digests; normalized WER 9.4595% both | The historical **12B Unified base-checkpoint** comparison used 100 samples per dataset: | Metric | Native preprocessing -> FP16 ONNX | HF preprocessing -> same ONNX | HF PyTorch BF16 | |---|---:|---:|---:| | AI2D strict single-digit accuracy | 50% | 51% | 50% | | AI2D evaluator-normalized accuracy | 61% | 59% | 59% | | FLEURS normalized WER (lower is better) | 18.13% | 17.24% | 18.59% | Qualifications for the historical 12B comparison: - The harness suppressed structural IDs 258882/258883 uniformly across all three generation paths. These are controlled quality comparisons under that shared search policy, not unmodified generation or exact-parity claims. - HF prompts explicitly included the media-boundary newlines that the native processor inserts automatically. Input IDs were not copied or overwritten to force agreement. - The BF16 Torch reference used eager attention. An eager-attention mask issue was found in the separate E2B investigation; the historical 12B reference still needs a backend cross-check before being treated as definitive. - Empty outputs and generation-cap hits were retained. These are fixed 100-sample subsets, not full-dataset release qualification. - Native/HF image preprocessing values differ; the known image discrepancy remains tracked in microsoft/onnxruntime-extensions#1131. **No completed CPU 100-AI2D/100-FLEURS quality result is claimed.** A newer CPU FP32 dataset attempt was stopped after identifying an HF/native media-prompt layout mismatch in the harness. Its preliminary output differences are excluded from the quality/regression conclusions. We are retaining the historical dataset results and separately labeling the CPU smoke and latest-head integration tests rather than presenting them as a new full CPU evaluation. ## Dependency / merge order and CI The current dependency is the official Microsoft Extensions repository at the landed-main commit `99fec2db1c0a17de60eb0f1c2353c91f8d5e22ca`. microsoft/onnxruntime-extensions#1132 was squash-merged at 2026-10-08 16:32:22 UTC. Its landed commit and the previously validated PR head `6147e57f0def4627fb712bca1850d39480e5382c` have identical Git trees. The dependency and component manifest were updated together; the original landed-pin build passed 36 focused tests, and the latest review-fix build passed 40 against the same final pin. The Extensions landed-commit dependency gate is satisfied. This PR still requires its normal reviews and required CI; no merge was performed. Local x86 results do not establish Windows ARM64 success: the DOUBLE dummy-vision change targets the observed ARM64 reduction discrepancy, which remote CI must confirm. Task-created large model/checkpoint/dataset/cache/build assets are cleaned up after preserving small reports, configuration snapshots, hashes, and reproduction evidence. No full-dataset, performance, or release requalification is claimed. --------- Signed-off-by: Justin Chu <[email protected]> Co-authored-by: Justin Chu <[email protected]> Co-authored-by: Ti-Tai Wang <[email protected]> Co-authored-by: Copilot <[email protected]>
1 parent 6649291 commit 0fb858c

9 files changed

Lines changed: 1381 additions & 135 deletions

File tree

‎cgmanifests/generated/cgmanifest.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@
4646
"component": {
4747
"type": "git",
4848
"git": {
49-
"commitHash": "75129dc9da91f95c22d3f83087571a0759ed0d5c",
49+
"commitHash": "99fec2db1c0a17de60eb0f1c2353c91f8d5e22ca",
5050
"repositoryUrl": "https://github.com/microsoft/onnxruntime-extensions.git"
5151
},
5252
"comments": "onnxruntime_extensions"

‎cmake/deps.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ pybind11;https://github.com/pybind/pybind11/archive/refs/tags/v2.13.6.zip;f78029
1414
googletest;https://github.com/google/googletest/archive/530d5c8c84abd2a46f38583ee817743c9b3a42b4.zip;5e3a61db2aa975cfd0f97ba92c818744e7fa7034
1515
microsoft_wil;https://github.com/microsoft/wil/archive/refs/tags/v1.0.230629.1.zip;e4a542a323c070376f7c2d1973d0f7ddbc1d2fa5
1616
directx_headers;https://github.com/microsoft/DirectX-Headers/archive/refs/tags/v1.613.1.zip;47653509a3371eabb156360f42faf582f314bf2e
17-
onnxruntime_extensions;https://github.com/microsoft/onnxruntime-extensions.git;0353c93d966d74706724e41ddab63f580eebbbf0
17+
onnxruntime_extensions;https://github.com/microsoft/onnxruntime-extensions.git;99fec2db1c0a17de60eb0f1c2353c91f8d5e22ca
1818

1919
# These two dependencies are for the optional constrained decoding feature (USE_GUIDANCE)
2020
llguidance;https://github.com/microsoft/llguidance.git;94fa39128ef184ffeda33845f6d333f332a34b4d

‎src/models/model_type.h‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ struct ModelType {
3636

3737
inline static bool IsMMM(const std::string& model_type) {
3838
// Multi-modal model (MMM)
39-
static constexpr std::array<std::string_view, 2> MMM = {"gemma4", "phi4mm"};
39+
static constexpr std::array<std::string_view, 3> MMM = {"gemma4", "gemma4_unified", "phi4mm"};
4040
return std::find(MMM.begin(), MMM.end(), model_type) != MMM.end();
4141
}
4242

‎src/models/preprocessing/gemma4_multimodal_processor.cpp‎

Lines changed: 65 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -188,9 +188,21 @@ ProcessGemma4Prompt(const Generators::Tokenizer& tokenizer, const std::string& p
188188

189189
Gemma4MultiModalProcessor::Gemma4MultiModalProcessor(Config& config, const SessionInfo& session_info)
190190
: pixel_values_type_{session_info.GetInputDataType(config.model.vision.inputs.pixel_values)} {
191+
// gemma-4-12B "unified" is encoder-free: it consumes raw 48px merged pixel
192+
// patches (patch_dim 6912) and raw 640-sample waveform frames directly. The
193+
// preprocessing ops (Gemma4ImageTransform at patch_size=48/pooling=1 and
194+
// Gemma4UnifiedAudioFrames) already emit the final contract, so — unlike the
195+
// SigLIP/log-mel "gemma4" path — no pixel trimming or conv-subsampling of the
196+
// audio-token count is applied.
197+
unified_ = config.model.type == "gemma4_unified";
198+
191199
// Query pixel_position_ids type (int32 or int64) if the vision model has this input
192200
if (session_info.HasInput(config.model.vision.inputs.pixel_position_ids)) {
193201
pixel_position_ids_type_ = session_info.GetInputDataType(config.model.vision.inputs.pixel_position_ids);
202+
if (pixel_position_ids_type_ != ONNX_TENSOR_ELEMENT_DATA_TYPE_INT32 &&
203+
pixel_position_ids_type_ != ONNX_TENSOR_ELEMENT_DATA_TYPE_INT64) {
204+
throw std::runtime_error("Gemma4 pixel_position_ids input must have type INT32 or INT64");
205+
}
194206
}
195207
const auto image_processor_config = (config.config_path / fs::path(config.model.vision.config_filename)).string();
196208
CheckResult(OrtxCreateProcessor(image_processor_.ToBeAssigned(), image_processor_config.c_str()));
@@ -298,14 +310,14 @@ std::unique_ptr<NamedTensors> Gemma4MultiModalProcessor::Process(const Tokenizer
298310
CheckResult(OrtxFeatureExtraction(audio_processor_.get(), payload.audios->audios_.get(), audio_result.ToBeAssigned()));
299311

300312
// OrtxTensorResultGetAt allocates a new TensorObject the caller owns; wrap in OrtxObjectPtr to dispose.
301-
ort_extensions::OrtxObjectPtr<OrtxTensor> audio_features_owner;
313+
ort_extensions::OrtxObjectPtr<OrtxTensor> audio_features_owner, audio_attention_mask_owner;
302314
CheckResult(OrtxTensorResultGetAt(audio_result.get(), 0, audio_features_owner.ToBeAssigned()));
315+
CheckResult(OrtxTensorResultGetAt(audio_result.get(), 1, audio_attention_mask_owner.ToBeAssigned()));
303316
OrtxTensor* audio_features = audio_features_owner.get();
317+
OrtxTensor* audio_attention_mask = audio_attention_mask_owner.get();
304318

305319
EmplaceProcessedTensor(*named_tensors, Config::Defaults::AudioEmbedsName, audio_features, audio_features_type_, allocator);
306320

307-
// Create input_features_mask: all-True for single-clip inference (no padding)
308-
// Shape matches audio features: [batch, time] bool
309321
const float* audio_data{};
310322
const int64_t* audio_shape{};
311323
size_t audio_dims;
@@ -319,17 +331,23 @@ std::unique_ptr<NamedTensors> Gemma4MultiModalProcessor::Process(const Tokenizer
319331
"but received " +
320332
std::to_string(batch_dim) + " clips.");
321333
}
322-
{
323-
auto mask = OrtValue::CreateTensor<bool>(allocator, std::vector<int64_t>{batch_dim, time_dim});
324-
std::fill_n(mask->GetTensorMutableData<bool>(), batch_dim * time_dim, true);
325-
named_tensors->emplace(std::string(Config::Defaults::AudioAttentionMaskName),
326-
std::make_shared<Tensor>(std::move(mask)));
334+
// Preserve the extractor's actual mask. Current single-clip frames are all
335+
// valid; retaining the mask also respects padding bounds if batching evolves.
336+
named_tensors->emplace(std::string(Config::Defaults::AudioAttentionMaskName),
337+
std::make_shared<Tensor>(ProcessTensor<bool>(audio_attention_mask, allocator)));
338+
339+
// Compute audio_sizes / audio-token count.
340+
// Unified: each 640-sample frame is exactly one audio soft token, so the
341+
// token count equals the number of frames (time_dim) — no subsampling.
342+
// Standard gemma4: the Conformer speech encoder applies 2-stage Conv2d with
343+
// stride=2 each, so the token count is reduced accordingly.
344+
if (unified_) {
345+
num_audio_tokens = time_dim;
346+
} else {
347+
int64_t t_after_1 = (time_dim - 1) / 2 + 1;
348+
int64_t t_after_2 = (t_after_1 - 1) / 2 + 1;
349+
num_audio_tokens = t_after_2;
327350
}
328-
329-
// Compute audio_sizes: the speech encoder uses 2-stage Conv2d with stride=2 each
330-
int64_t t_after_1 = (time_dim - 1) / 2 + 1;
331-
int64_t t_after_2 = (t_after_1 - 1) / 2 + 1;
332-
num_audio_tokens = t_after_2;
333351
std::array<int64_t, 1> audio_sizes_shape = {1};
334352
auto audio_sizes = OrtValue::CreateTensor<int64_t>(allocator, audio_sizes_shape);
335353
audio_sizes->GetTensorMutableData<int64_t>()[0] = num_audio_tokens;
@@ -345,8 +363,15 @@ std::unique_ptr<NamedTensors> Gemma4MultiModalProcessor::Process(const Tokenizer
345363

346364
if (payload.images) {
347365
// The Gemma4ImageTransform pads pixel_values and position_ids to max_patches.
348-
// The vision ONNX model expects the actual (unpadded) number of patches.
349-
// Trim the tensors to actual_patches = actual_soft_tokens * pooling_kernel_size².
366+
//
367+
// Standard gemma4: the SigLIP vision ONNX expects the actual (unpadded)
368+
// number of teacher patches, so the tensors are trimmed to
369+
// actual_patches = actual_soft_tokens * pooling_kernel_size².
370+
//
371+
// Unified (gemma4_unified): the encoder-free vision graph consumes the full
372+
// padded (max_soft_tokens, 6912) tensors and strips padding internally using
373+
// position_ids (== -1 marks padding). No trimming is applied; actual_patches
374+
// is computed but the trim branches below are gated on !unified_.
350375
constexpr int64_t kPoolingKernelSize = 3;
351376
const int64_t actual_patches = static_cast<int64_t>(max_soft_tokens) * kPoolingKernelSize * kPoolingKernelSize;
352377

@@ -361,7 +386,7 @@ std::unique_ptr<NamedTensors> Gemma4MultiModalProcessor::Process(const Tokenizer
361386
const int64_t num_padded_patches = (pv_dims == 3) ? pv_shape[1] : pv_shape[0];
362387
const int64_t patch_dim = (pv_dims == 3) ? pv_shape[2] : pv_shape[1];
363388

364-
if (actual_patches < num_padded_patches) {
389+
if (!unified_ && actual_patches < num_padded_patches) {
365390
// Trim: copy only the first actual_patches from the padded tensor.
366391
// The preprocessor outputs float32 data, so we trim using float32 strides,
367392
// then cast to the model's pixel_values type (float, fp16, or bf16).
@@ -406,33 +431,34 @@ std::unique_ptr<NamedTensors> Gemma4MultiModalProcessor::Process(const Tokenizer
406431
const int64_t num_padded_pos = (pos_dims == 3) ? pos_shape[1] : pos_shape[0];
407432
const int64_t pos_last_dim = (pos_dims == 3) ? pos_shape[2] : pos_shape[1];
408433

409-
if (actual_patches < num_padded_pos) {
410-
// Trim position_ids: for 3D, copy per-batch with correct stride.
411-
// Detect the element type from the vision model's input to handle both int32 and int64.
412-
const int64_t pos_batch = (pos_dims == 3) ? pos_shape[0] : 1;
413-
auto trimmed_pos_shape = (pos_dims == 3) ? std::vector<int64_t>{pos_batch, actual_patches, pos_last_dim}
414-
: std::vector<int64_t>{actual_patches, pos_last_dim};
415-
416-
auto trimmed_pos = OrtValue::CreateTensor(allocator, trimmed_pos_shape, pixel_position_ids_type_);
417-
const size_t pos_elem_size = (pixel_position_ids_type_ == ONNX_TENSOR_ELEMENT_DATA_TYPE_INT32) ? 4 : 8;
418-
auto* dst = static_cast<uint8_t*>(trimmed_pos->GetTensorMutableRawData());
419-
const auto* src = static_cast<const uint8_t*>(pos_data_raw);
420-
const size_t src_stride = static_cast<size_t>(num_padded_pos * pos_last_dim) * pos_elem_size;
421-
const size_t dst_stride = static_cast<size_t>(actual_patches * pos_last_dim) * pos_elem_size;
434+
// Gemma4ImageTransform always emits INT64 positions, regardless of the
435+
// graph's requested type. Use source-element strides for both trimmed
436+
// standard grids and full padded unified grids, converting values rather
437+
// than reinterpreting the source bytes when the graph expects INT32.
438+
const int64_t pos_batch = (pos_dims == 3) ? pos_shape[0] : 1;
439+
const int64_t output_patches = (!unified_ && actual_patches < num_padded_pos)
440+
? actual_patches
441+
: num_padded_pos;
442+
auto output_pos_shape = (pos_dims == 3) ? std::vector<int64_t>{pos_batch, output_patches, pos_last_dim}
443+
: std::vector<int64_t>{output_patches, pos_last_dim};
444+
auto output_pos = OrtValue::CreateTensor(allocator, output_pos_shape, pixel_position_ids_type_);
445+
const auto* src = static_cast<const int64_t*>(pos_data_raw);
446+
const size_t src_stride = static_cast<size_t>(num_padded_pos * pos_last_dim);
447+
const size_t dst_stride = static_cast<size_t>(output_patches * pos_last_dim);
448+
if (pixel_position_ids_type_ == ONNX_TENSOR_ELEMENT_DATA_TYPE_INT32) {
449+
auto* dst = output_pos->GetTensorMutableData<int32_t>();
422450
for (int64_t b = 0; b < pos_batch; ++b) {
423-
std::memcpy(dst + b * dst_stride, src + b * src_stride, dst_stride);
451+
std::transform(src + b * src_stride, src + b * src_stride + dst_stride,
452+
dst + b * dst_stride, [](int64_t value) { return static_cast<int32_t>(value); });
424453
}
425-
named_tensors->emplace(std::string(Config::Defaults::PixelPositionIdsName),
426-
std::make_shared<Tensor>(std::move(trimmed_pos)));
427-
} else {
428-
if (pixel_position_ids_type_ == ONNX_TENSOR_ELEMENT_DATA_TYPE_INT32) {
429-
named_tensors->emplace(std::string(Config::Defaults::PixelPositionIdsName),
430-
std::make_shared<Tensor>(ProcessTensor<int32_t>(pixel_position_ids, allocator)));
431-
} else {
432-
named_tensors->emplace(std::string(Config::Defaults::PixelPositionIdsName),
433-
std::make_shared<Tensor>(ProcessTensor<int64_t>(pixel_position_ids, allocator)));
454+
} else if (pixel_position_ids_type_ == ONNX_TENSOR_ELEMENT_DATA_TYPE_INT64) {
455+
auto* dst = output_pos->GetTensorMutableData<int64_t>();
456+
for (int64_t b = 0; b < pos_batch; ++b) {
457+
std::copy_n(src + b * src_stride, dst_stride, dst + b * dst_stride);
434458
}
435459
}
460+
named_tensors->emplace(std::string(Config::Defaults::PixelPositionIdsName),
461+
std::make_shared<Tensor>(std::move(output_pos)));
436462
}
437463
}
438464

‎src/models/preprocessing/gemma4_multimodal_processor.h‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ struct Gemma4MultiModalProcessor : Processor {
2020
ONNXTensorElementDataType audio_features_type_;
2121

2222
bool has_speech_{false};
23+
bool unified_{false}; // gemma-4-12B encoder-free "unified" variant
2324
size_t vision_soft_tokens_per_image_{260};
2425
};
2526

‎src/models/preprocessing/multi_modal_processor.cpp‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ MultiModalProcessor::MultiModalProcessor(Config& config, const SessionInfo& sess
3636
{"lfm2_audio", Processor::Create<Lfm2AudioProcessor>},
3737
{"lfm2_vl", Processor::Create<Lfm2VlImageProcessor>},
3838
{"gemma4", Processor::Create<Gemma4MultiModalProcessor>},
39+
{"gemma4_unified", Processor::Create<Gemma4MultiModalProcessor>},
3940
{"mistral3", Processor::Create<Mistral3ImageProcessor>},
4041
{"fara", Processor::Create<QwenImageProcessor>},
4142
{"qwen2_5_vl", Processor::Create<QwenImageProcessor>},

0 commit comments

Comments
 (0)