fix(media-understanding): video auto-selection runs without a model by zhangguiping-xydt · Pull Request #99791 · openclaw/openclaw

zhangguiping-xydt · 2026-07-04T03:29:10Z

What Problem This Solves

Fixes an issue where users relying on automatic video media understanding could select a video-capable provider without forwarding that provider's declared default video model.

Why This Change Was Made

Video auto-selection now resolves defaultModels.video for both active and key-selected providers when no explicit model is configured. Providers without default-model metadata retain the existing optional-model contract and can continue to choose their own default inside describeVideo.

The change stays within media-understanding provider entry resolution. It does not change provider metadata, authentication, explicit model precedence, schemas, or provider catalog defaults.

User Impact

Video attachments can be described through auto-selected providers such as Moonshot with the provider's declared default model. Existing explicit video model configuration still wins, and custom self-defaulting video providers remain eligible with model unset.

Evidence

Focused regression suite:

node scripts/run-vitest.mjs \
  src/media-understanding/runner.video.test.ts \
  src/media-understanding/runner.auto-audio.test.ts

Test Files  2 passed (2)
Tests       14 passed (14)

The regression coverage proves:
- active video providers receive defaultModels.video;
- key-selected video providers receive defaultModels.video;
- explicit video models retain precedence;
- providers without model metadata are invoked with model: undefined and may self-default;
- adjacent audio model gating remains unchanged.

Focused formatting and patch checks passed:

node_modules/.bin/oxfmt --check \
  src/media-understanding/runner.ts \
  src/media-understanding/runner.video.test.ts
git diff --check

Fresh branch autoreview against current origin/main found no actionable issues.

clawsweeper · 2026-07-04T03:34:46Z

Codex review: needs maintainer review before merge. Reviewed July 4, 2026, 3:03 AM ET / 07:03 UTC.

Summary
The PR resolves default video models for auto-selected active/key video providers and adds video runner tests for default-model use and no-default skip behavior.

PR surface: Source +13, Tests +118. Total +131 across 2 files.

Reproducibility: yes. Source inspection of current main shows video auto-selection can build a provider entry with model unset, and runProviderEntry passes that unset value into describeVideo.

Review metrics: 1 noteworthy metric.

Video auto-selection gates: 2 resolver paths changed. Active-provider and key-provider video auto-selection now require a resolvable model, which is the compatibility-sensitive behavior maintainers need to notice.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

[P2] Have a maintainer explicitly accept the strict video default-model requirement or request the compatibility path before merge.

Risk before merge

[P1] Merging changes auto-selected video providers that expose describeVideo but omit defaultModels.video from invoked-with-undefined-model to skipped, so custom providers may need metadata or explicit video model configuration.

Maintainer options:

Accept the strict video model gate (recommended)
Maintainers can accept that auto-selected video providers must declare defaultModels.video or rely on explicit video model configuration before merge.
Preserve a compatibility path
If no-default video providers are intended to remain auto-selectable, adjust the resolver contract so they stay eligible only through a documented self-default or explicit configuration path.
Pause for provider contract direction
If the provider metadata requirement is not settled, pause this PR until maintainers decide the permanent video auto-selection contract.

Next step before merge

[P2] Manual review is needed because the remaining blocker is provider contract acceptance or compatibility-path direction, not a narrow automated repair.

Security
Cleared: The diff changes media-understanding resolver logic and tests only; no concrete security or supply-chain concern was found.

Review details

Best possible solution:

Land the resolver-boundary fix after maintainers accept the strict video auto-selection contract, or add a documented compatibility path for no-default video providers before merge.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection of current main shows video auto-selection can build a provider entry with model unset, and runProviderEntry passes that unset value into describeVideo.

Is this the best way to solve the issue?

Yes, with maintainer acceptance. Resolving default video models at the resolver boundary is narrower than adding provider-specific execution fallbacks, but skipping no-default providers is a compatibility-sensitive contract choice.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against f48ff25b3be3.

Label changes

Label justifications:

P2: The PR fixes a real media-understanding provider/model routing bug with limited blast radius.
merge-risk: 🚨 compatibility: The diff changes upgrade behavior for auto-selected video providers that expose describeVideo but do not declare a video default model.
rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (logs): The PR body includes redacted after-fix runtime output showing the selected Moonshot video default model reaches the provider request and no-default providers are skipped.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes redacted after-fix runtime output showing the selected Moonshot video default model reaches the provider request and no-default providers are skipped.

Evidence reviewed

PR surface:

Source +13, Tests +118. Total +131 across 2 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	1	17	4	+13
Tests	1	122	4	+118
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	2	139	8	+131

What I checked:

Repository policy: Root AGENTS.md was read fully; its ClawSweeper review policy treats provider routing, defaults, fallback behavior, and compatibility-sensitive provider/plugin behavior as maintainer-visible merge risk. (AGENTS.md, f48ff25b3be3)
Current main key-provider path: On current main, resolveKeyEntry only resolves/requires a model for image; non-image capabilities, including video, return the passed model value, which is undefined for key-selected auto providers. (src/media-understanding/runner.ts:626, f48ff25b3be3)
Current main active-provider path: On current main, active video providers use params.activeModel?.model and only image/audio reject a missing model, so active video can continue with model unset. (src/media-understanding/runner.ts:894, f48ff25b3be3)
Execution path: runProviderEntry builds the video request with model: entry.model and returns result.model ?? entry.model, so an unset resolver model reaches describeVideo and the output metadata. (src/media-understanding/runner.entries.ts:900, f48ff25b3be3)
Provider contract: MediaUnderstandingProvider makes defaultModels optional, so requiring defaultModels.video for auto-selected video providers is a compatibility-sensitive contract decision for custom providers. (src/media-understanding/types.ts:247, f48ff25b3be3)
Official video provider defaults: The built-in Moonshot, Google, and Qwen video media-understanding providers declare defaultModels.video, so the stricter resolver path is covered for these official providers. (extensions/moonshot/media-understanding-provider.ts:82, f48ff25b3be3)

Likely related people:

steipete: Authored the media auto-detect runner expansion, moved provider defaults into media metadata, and added Qwen video media-understanding support that shares the default-model contract. (role: feature history owner; confidence: high; commits: 2dfbd1c1f6a6, 3a3f88a80a78, e3ac0f43df3e; files: src/media-understanding/runner.ts, src/media-understanding/defaults.ts, extensions/qwen/media-understanding-provider.ts)
shakkernerd: Authored the configured media auth provider ordering change in the resolver path that now participates in auto-selected provider behavior. (role: recent adjacent contributor; confidence: medium; commits: 14ff2c30d118; files: src/media-understanding/runner.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

vincentkoc · 2026-07-06T19:28:19Z

@clawsweeper re-review

clawsweeper · 2026-07-06T19:28:22Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

vincentkoc · 2026-07-06T19:41:29Z

Land-ready maintainer verification completed for exact head 085cb3fb19dea9916139f38c90edc48b462390fe.

The maintainer repair preserves video providers that intentionally self-default while still forwarding defaultModels.video for active and key-selected providers. Explicit model precedence and the existing audio guard remain unchanged.

Validation:

node scripts/run-vitest.mjs src/media-understanding/runner.video.test.ts src/media-understanding/runner.auto-audio.test.ts - 2 files, 14 tests passed.
node_modules/.bin/oxfmt --check src/media-understanding/runner.ts src/media-understanding/runner.video.test.ts - passed.
git diff --check - passed.
Fresh branch autoreview against origin/main - no actionable findings.
Exact-head CI run 28817459029 - success, including QA Smoke and all 44 executed jobs.
OPENCLAW_TESTBOX=1 scripts/pr prepare-run 99791 - passed after refreshing current origin/main; prepared head equals PR head.

Known proof gaps: none for this resolver-level change.

vincentkoc · 2026-07-06T19:42:06Z

Merged via squash.

Prepared head SHA: 085cb3fb19dea9916139f38c90edc48b462390fe
Landed commit: 565097fc1001b26b36e98458779ac34d7ea43e06

…penclaw#99791) * fix: require video media provider models * fix video media test typing and skip coverage * fix(media-understanding): preserve self-defaulting video providers --------- Co-authored-by: Vincent Koc <[email protected]>

fix: require video media provider models

22eec3d

openclaw-barnacle Bot added the size: S label Jul 4, 2026

zhangguiping-xydt changed the title ~~media understanding video provider selection must require a model~~ fix: video media understanding runs without a selected model Jul 4, 2026

fix video media test typing and skip coverage

e881979

zhangguiping-xydt changed the title ~~fix: video media understanding runs without a selected model~~ fix(media-understanding): video auto-selection runs without a model Jul 4, 2026

fix(media-understanding): preserve self-defaulting video providers

085cb3f

vincentkoc self-assigned this Jul 6, 2026

vincentkoc merged commit 565097f into openclaw:main Jul 6, 2026
104 checks passed

github-actions Bot mentioned this pull request Jul 6, 2026

📡 Upstream Digest — 2026-07-06 19:44 UTC curtismercier/openclaw-mods#1224

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(media-understanding): video auto-selection runs without a model#99791

fix(media-understanding): video auto-selection runs without a model#99791
vincentkoc merged 3 commits into
openclaw:mainfrom
zhangguiping-xydt:fix/problem-media-video-model-required

zhangguiping-xydt commented Jul 4, 2026 •

edited by vincentkoc

Loading

Uh oh!

clawsweeper Bot commented Jul 4, 2026 •

edited

Loading

Uh oh!

vincentkoc commented Jul 6, 2026

Uh oh!

clawsweeper Bot commented Jul 6, 2026 •

edited

Loading

Uh oh!

vincentkoc commented Jul 6, 2026

Uh oh!

Uh oh!

vincentkoc commented Jul 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

zhangguiping-xydt commented Jul 4, 2026 • edited by vincentkoc Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Problem This Solves

Why This Change Was Made

User Impact

Evidence

Uh oh!

clawsweeper Bot commented Jul 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vincentkoc commented Jul 6, 2026

Uh oh!

clawsweeper Bot commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vincentkoc commented Jul 6, 2026

Uh oh!

Uh oh!

vincentkoc commented Jul 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

zhangguiping-xydt commented Jul 4, 2026 •

edited by vincentkoc

Loading

clawsweeper Bot commented Jul 4, 2026 •

edited

Loading

clawsweeper Bot commented Jul 6, 2026 •

edited

Loading