Skip to content

fix(vllm): put omni video negative_prompt in prompt dict entry (fixes #15349) - #15402

Merged
rmccorm4 merged 3 commits into
ai-dynamo:mainfrom
charan-rathore:fix/video-negative-prompt
Oct 2, 2026
Merged

rmccorm4 merged 3 commits into
ai-dynamo:mainfrom
charan-rathore:fix/video-negative-prompt

Conversation

@charan-rathore

@charan-rathore charan-rathore commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #15349.

Problem

A vLLM-Omni video generation request with nvext.negative_prompt set returns HTTP 500. In OmniHandler, prompt is a dict-shaped OmniTextPrompt, but the code assigns prompt.negative_prompt = nvext.negative_prompt as an attribute, which fails for this prompt type. The crash happens while the request is being prepared, before the engine starts.

Change

One production line: put the negative prompt into the existing "negative_prompt" dict entry instead. Requests without a negative prompt are unchanged, and an optional starting image still flows through.

Adds five regression test cases to the existing video request tests: absent/empty/ordinary/non-English negative prompts, plus one image co-existence case.

Validation

Ran on this commit (base 2de120f), CPU only:

  • 4 failures / 5 passes before the fix, 9/9 passes after (same runner, same environment). The runner executes the checkout's actual request-builder method and EngineInputs, loads the actual Pydantic video request models, and runs the new regression test method; it uses a dict-shaped stand-in for OmniTextPrompt and stand-ins for the sampling/LoRA helpers, with heavy imports bypassed via AST extraction.
  • Lint with the repo-pinned tools is clean on the changed files: ruff check (0.5.2), black --check (23.1.0), isort --check-only (5.12.0).

No native vLLM-Omni test suite, GPU inference, generated video, or Docker run was performed.

AI disclosure: this PR was prepared with AI assistance (Instinct assisted) under my direction; I understand the change and take responsibility for it.

@charan-rathore
charan-rathore requested review from a team as code owners September 30, 2026 04:25
@copy-pr-bot

copy-pr-bot Bot commented Sep 30, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions

Copy link
Copy Markdown
Contributor

👋 Hi charan-rathore! Thank you for contributing to ai-dynamo/dynamo.

Just a reminder: The NVIDIA Test Github Validation CI runs an essential subset of the testing framework to quickly catch errors.Your PR reviewers may elect to test the changes comprehensively before approving your changes.

🚀

@github-actions github-actions Bot added external-contribution Pull request is from an external contributor fix backend::vllm Relates to the vllm backend multimodal labels Sep 30, 2026
@coderabbitai

coderabbitai Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: ai-dynamo/dynamo/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: b36b5509-7c1a-43c6-bb6f-56ccd9d6c997

📥 Commits

Reviewing files that changed from the base of the PR and between 2de120f and f8a2ffe.

📒 Files selected for processing (2)
  • components/src/dynamo/vllm/omni/omni_handler.py
  • components/src/dynamo/vllm/tests/omni/test_omni_handler.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.


Walkthrough

The video request builder now stores negative_prompt as a dictionary key. Parameterized tests cover absent, empty, plain-text, and Unicode values, with and without an input image, and verify the request type and response format.

Changes

Video Prompt Handling

Layer / File(s) Summary
Store and verify video negative prompts
components/src/dynamo/vllm/omni/omni_handler.py, components/src/dynamo/vllm/tests/omni/test_omni_handler.py
The video request builder stores negative_prompt as a dictionary entry. Parameterized tests check prompt values, optional image data, request type, and response format.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~8 minutes

Severity of issue fixed: Medium

Merge Risk: ⚪ Minimal · up to f8a2f

Negative prompts are now passed in the prompt mapping, with focused coverage of the request variants. No concrete defect or required-validation omission is established, so the change has minimal merge risk.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 2 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The change directly addresses #15349. _engine_inputs_from_video now stores a non-None nvext.negative_prompt in prompt["negative_prompt"], which matches the dict-shaped prompt expected by the eng…
Out of Scope Changes check ✅ Passed The pull request changes only the Omni video request builder and adds focused regression tests for the linked negative-prompt failure. The changes stay within #15349. No unrelated production behavior …
Title check ✅ Passed The title clearly and concisely describes the primary fix: storing the Omni video negative prompt in the prompt dictionary entry.
Description check ✅ Passed The description explains the problem, implementation, validation, limitations, and linked issue. It does not include the requested “Where should the reviewer start?” section, but the description is ot…
  • Fix all pre-merge checks with AI

Comment @coderabbitai help to get the list of available commands.

Keep the minimal negative-prompt cases (absent, empty, ordinary,
non-English) and one image co-existence case instead of crossing every
value with both image states, per review feedback.

Signed-off-by: Charan Rathore <[email protected]>
@charan-rathore

Copy link
Copy Markdown
Contributor Author

Good point, done in 0ee4917. The parametrization is consolidated to the minimal negative-prompt cases (absent, empty, ordinary, non-English) plus a single image co-existence case, instead of crossing every value with both image states. Image handling remains covered by test_t2v_no_multi_modal_data_and_i2v_attaches_image. Same runner re-verified: 4 failures / 5 passes without the fix, 9/9 passes with it, and the PR body is updated to match.

@charan-rathore

Copy link
Copy Markdown
Contributor Author

Friendly check-in: this one is ready from my side, and the review notes are addressed in 0ee4917. Is there anything else you need from me to move it toward review? #15541 builds on this fix, so it would unblock that one too.

@rmccorm4
rmccorm4 temporarily deployed to external_collaborator October 2, 2026 01:52 — with GitHub Actions Inactive
@rmccorm4

rmccorm4 commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

/ok to test cb172e5

@rmccorm4
rmccorm4 enabled auto-merge (squash) October 2, 2026 01:52
@rmccorm4
rmccorm4 merged commit 317777c into ai-dynamo:main Oct 2, 2026
108 checks passed
@charan-rathore
charan-rathore deleted the fix/video-negative-prompt branch October 2, 2026 02:57

This branch was previously deployed

1 inactive deployment
external_collaborator — cb172e50 Deployed Oct 2, 2026 by rmccorm4 via ok-to-test #25087
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend::vllm Relates to the vllm backend external-contribution Pull request is from an external contributor fix multimodal size/M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG]: vLLM-Omni video generation returns HTTP 500 with nvext.negative_prompt

3 participants