Skip to content

epp: parse multipart video requests for routing - #2937

Merged
llm-d-router-release-notes[bot] merged 13 commits into
llm-d:mainfrom
0z5a:work/a2-video-parser
Nov 2, 2026
Merged

llm-d-router-release-notes[bot] merged 13 commits into
llm-d:mainfrom
0z5a:work/a2-video-parser

Conversation

@0z5a

@0z5a 0z5a commented Sep 20, 2026 •

Copy link
Copy Markdown
Contributor

/kind feature

What this PR does / why we need it:

The EPP OpenAI parser claims POST /v1/videos and POST /v1/videos/sync. It reads the model and scheduling fields from multipart forms so video generation requests can be routed. Video and image edits share one multipart field reader. The original multipart bytes are forwarded as RawPayload.

This revision addresses review #5287195121: it removes MultipartPayload and video model rewriting, and places the video, registry, and forwarding tests in the existing package test files.

Which issue(s) this PR fixes:

Related to #1935 (multipart and video parser stretch goal).

Validation:

  • Targeted parser, registry, and director tests: go test -count=1 ./pkg/epp/framework/plugins/requesthandling/parsers/openai ./pkg/epp/handlers ./pkg/epp/requestcontrol.
  • go build ./... and go vet ./pkg/epp/framework/plugins/requesthandling/parsers/openai.
  • Live E2E through Envoy -> EPP -> an existing vLLM Omni Wan2.2 backend: synchronous generation returned HTTP 200 and a 1,112,446-byte MP4; asynchronous creation, status polling, content retrieval, and deletion completed. Direct backend, synchronous routed, and asynchronous routed outputs had the same SHA-256 (2dd31c443f054b0e058a240c3e65c6b5141b65467f489b6afb8ee1dfc9767d02). Each decoded as H.264, 832x480, 9 frames.

go test -count=1 ./pkg/epp/... has one failure in TestGenerateRequest_UnmarshalJSON/invalid_placeholder_preserves_error_field. The same failure reproduces on the pristine previous PR head (032ea03e) with Go 1.27.1: the test expects an older JSON error string. All other EPP packages passed.

CPU benchmark (1 MiB multipart media, Intel Xeon Gold 6530, CPU 80, GOMAXPROCS=1, five 200-operation samples per round; medians shown):

Path Previous PR head This revision Speedup Allocations/op
Video parse and forwarding decision, round 1 1.562 ms 0.664 ms 2.35x 221 -> 116
Video parse and forwarding decision, round 2 (reverse run order) 1.262 ms 0.661 ms 1.91x 221 -> 116
Video parsing only 0.613 ms 0.661 ms 0.93x 121 -> 115
Image edits parsing only 0.587 ms 0.646 ms 0.91x 51 -> 50

The forwarding benchmark uses the same multipart request on both revisions and applies the director's payload decision for a different target model. The previous branch rewrites and marshals the video body; this revision forwards RawPayload. It measures router CPU work, not video generation time. The shared parser adds some parse-only time, as shown in the last two rows.

Release note:

The EPP parses multipart video generation requests for routing and forwards their original bodies unchanged.

The OpenAI parser claimed no video path, so a video request fell through to
passthrough-parser: no model was extracted for routing and the body stayed
opaque, while the pinned vLLM-Omni backend serves both endpoints from one
multipart form handler.

Claims both video suffixes and parses the shared form, extracting the model
and the scheduler-relevant cost fields (steps, size, dimensions, frames, fps,
count). Media parts and the JSON-encoded reference fields are left in the raw
payload so the forwarded body is byte-identical to the request, and file parts
are drained to io.Discard rather than read into a buffer. Field bounds mirror
the backend's own, so overflow and malformed numbers fail at the router instead
of being clamped or zeroed into the cost fields.

See ws/A2/backend_contract.md for the source-derived wire contract.

Not included: rewriting the model field inside a multipart body. The router
forwards the multipart bytes verbatim, so RewriteModelName has no marshalable
payload to edit on this path; routing by the extracted model works, rewriting
does not.

Signed-off-by: 0z5a <[email protected]>
The video parser forwarded the body as raw bytes, which is not marshalable, so
the director skipped the model rewrite on /v1/videos and /v1/videos/sync and the
model the backend validated could differ from the one admission used.

The payload is now a MultipartPayload that keeps the received body and splices a
replacement into the last model form field, which is the field the backend
reads. Every other part is copied verbatim, so media bytes, part headers, and
the boundary lines are unchanged.

The integer form fields accept the surrounding whitespace and integral float
spellings the backend accepts, so the router no longer rejects a request the
backend serves.

Signed-off-by: 0z5a <[email protected]>
@github-actions github-actions Bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. area/epp area/scheduling labels Sep 20, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🚨 Unsigned commits detected! Please sign your commits.

For instructions on how to set up GPG/SSH signing and verify your commits, please see GitHub Documentation.

@0z5a
0z5a force-pushed the work/a2-video-parser branch from f8e5cef to 698a32a Compare September 20, 2026 03:29
golangci-lint's goconst flags the multipart form field names that the
images-edits parser, the video parser and RewriteModelName all spell out.
Declare model and num_inference_steps once and use them at every site, and do
the same for the test fixture's default media type.

No behaviour change: the literals are identical, gofmt realigned the const
block, and the focused parser tests and go build still pass.

Signed-off-by: 0z5a <[email protected]>
@0z5a
0z5a force-pushed the work/a2-video-parser branch 2 times, most recently from ad26d3d to 032ea03 Compare September 20, 2026 06:12
@0z5a

0z5a commented Sep 21, 2026

Copy link
Copy Markdown
Contributor Author

Update: the last NOT_RUN item is now measured. Live-backend re-probe of the rewritten multipart body against a real vLLM-Omni server: 200 + video/mp4 (2,380,503 B, 7.469 s, X-Model echoing the served path), while the un-rewritten body is rejected by the backend's model check. Byte-identical outside the last model part (single contiguous change, length delta 52 = 78 − 26, 1 MiB media at the same offset with the same sha256); last-wins settled in both directions (served-then-alias → 400 naming the alias). An independently encoded curl request produced a bit-identical video (three-way sha256 match). Validation row A2-U06b live added above. Marking ready for review.

@0z5a
0z5a marked this pull request as ready for review September 21, 2026 01:25
@0z5a
0z5a requested review from a team and zetxqx as code owners September 21, 2026 01:25

@zetxqx zetxqx left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this, we don't need to support rewrite for video API so the new multiPartPaylaod is not needed here. Please convert it directly to rawPaylaod.

// request. Only scheduler-relevant scalar fields are read; the media parts and the
// JSON-encoded reference fields stay in Payload, which keeps the received bytes so a
// model rewrite can replace the model part alone and still forward the media verbatim.
func parseVideosRequest(body []byte, headers map[string]string) (*fwkrh.ParseResult, error) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a lot of common parts of multipart parsing with image edits multipart

// parseImagesEditsRequest parses a multipart/form-data /v1/images/edits request.
func parseImagesEditsRequest(body []byte, headers map[string]string) (*fwkrh.ParseResult, error) {
contentTypeValue, _ := headerValue(headers, contentType)
mediaType, params, err := mime.ParseMediaType(contentTypeValue)
if err != nil || mediaType != "multipart/form-data" {
return nil, errors.New("images edits request must have a multipart/form-data content-type")
}
boundary := params["boundary"]
if boundary == "" {
return nil, errors.New("images edits request: content-type is missing the multipart boundary")
}
images := &fwkrh.ImagesGenerationsRequest{}
extractedBody := &fwkrh.InferenceRequestBody{
Images: images,
Payload: fwkrh.RawPayload(body),
}
reader := multipart.NewReader(bytes.NewReader(body), boundary)
for {
part, err := reader.NextPart()
if errors.Is(err, io.EOF) {
break
}
if err != nil {
return nil, fmt.Errorf("error reading images edits multipart body: %w", err)
}
if part.FileName() != "" {
continue
}
value, err := io.ReadAll(part)
if err != nil {
return nil, fmt.Errorf("error reading images edits form field %q: %w", part.FormName(), err)
}
switch part.FormName() {
case "model":
extractedBody.Model = string(value)
case "prompt":
images.Prompt = string(value)
case "size":
images.Size = string(value)
case "n":
n, err := strconv.ParseInt(string(value), 10, 64)
if err != nil {
return nil, fmt.Errorf("invalid images edits n field: %w", err)
}
images.N = &n
case "num_inference_steps":
steps, err := strconv.ParseInt(string(value), 10, 64)
if err != nil {
return nil, fmt.Errorf("invalid images edits num_inference_steps field: %w", err)
}
images.NumInferenceSteps = &steps
case "stream":
stream, err := strconv.ParseBool(string(value))
if err != nil {
return nil, fmt.Errorf("invalid images edits stream field: %w", err)
}
extractedBody.Stream = stream
}
}
if images.Prompt == "" {
return nil, errors.New("invalid images edits request: must have prompt field")
}
return &fwkrh.ParseResult{Body: extractedBody, SkipResponseProcessing: false}, nil
}

can you extract the common part instead of repeating?

// so the parts the router does not interpret, media uploads in particular, keep the
// bytes, the headers, and the boundary lines the client sent. A payload with nothing
// to rewrite marshals to the received body itself.
type MultipartPayload struct {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see you want to add a new MultipartPayload to support modelRewrite. However in vllmomni doc: https://docs.vllm.ai/projects/vllm-omni/en/latest/serving/videos_api/#start-the-server the model in the video request is not needed.

So we can follow what we already have similar to imageEdits parsing the paylaod to rawPayload

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the test name is confusing. We should add this in openai_test.go

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test file name is confusing

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test file name is confusing

@0z5a
0z5a marked this pull request as draft September 23, 2026 06:47
@github-actions github-actions Bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Sep 23, 2026
@0z5a 0z5a changed the title epp: parse and rewrite multipart /v1/videos and /v1/videos/sync requests epp: parse multipart video requests for routing Sep 23, 2026
@github-actions github-actions Bot added the kind/feature Categorizes issue or PR as related to a new feature. label Sep 23, 2026
Reuse the multipart field reader with image edits and preserve raw video payloads for backend forwarding.

Signed-off-by: 0z5a <[email protected]>
@0z5a
0z5a force-pushed the work/a2-video-parser branch from f2b60b9 to 4c4acd5 Compare September 23, 2026 06:54
@0z5a

0z5a commented Sep 24, 2026

Copy link
Copy Markdown
Contributor Author

Addressed in 4c4acd5: video multipart requests now use RawPayload without model rewriting; parsing is shared with image edits, and the tests are in the existing OpenAI, handler, and director suites. Targeted Go tests, build/vet, and live sync/async E2E passed. The speed comparison is in the PR description. Thanks for the review.

@zetxqx

zetxqx commented Sep 24, 2026

Copy link
Copy Markdown
Contributor

@0z5a is this ready for review? I saw it is in draft still.

@0z5a
0z5a marked this pull request as ready for review September 25, 2026 09:44
@github-actions github-actions Bot added the needs-rebase PR has merge conflicts and needs a rebase label Oct 8, 2026
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

@0z5a This PR has merge conflicts with main. Please rebase on main and push to resolve them:

gh pr checkout 2937
git fetch origin main
git rebase origin/main
# resolve conflicts
git push --force-with-lease

@github-actions github-actions Bot removed the needs-rebase PR has merge conflicts and needs a rebase label Oct 11, 2026

@ahg-g ahg-g left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

zetxqx's earlier comments are addressed, and the parsing and forwarding look correct. Two inline: stale bound comments in types.go, and a note on async video polling with more than one replica.

// VideoGenerationRequest represents the OpenAI-compatible vLLM-Omni
// /v1/videos and /v1/videos/sync request body. Both endpoints are served by the
// same handler and accept multipart/form-data, so they share this shape.
// Field bounds mirror the pinned backend contract; omitted fields stay nil

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Drop the backend bounds from the VideoGenerationRequest comments: "Field bounds mirror the pinned backend contract" (L636), "at least 1" (L647), "1..200" (L657) and "1..10".

Why: e98ff34 removed the router-side bounds checks, as zetxqx asked on 10-08, so these comments now describe validation the router doesn't do → readers will assume out-of-range values are rejected before reaching vLLM-Omni.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 29f416e. Removed the backend bound claims from the VideoGenerationRequest comments (Width/Height, NumInferenceSteps, NumOutputsPerPrompt). The nil-handling note stays.

imagesEditsAPI = "images/edits"
// videosAPI is the OpenAI-compatible vLLM-Omni asynchronous video generation
// endpoint. Requests are multipart/form-data.
videosAPI = "videos"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Document that async /v1/videos needs a single-replica pool, or session affinity, for status and content requests.

Why: the create POST is routed by load, but GET /v1/videos/{id} and /content have no body, so the router sends them to a random pod (handlers/request.go:44) → with more than one vLLM-Omni replica, a poll lands on a pod that doesn't hold the job → status and content calls return 404. The E2E in the description used one backend, so it doesn't cover this. /v1/videos/sync isn't affected.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 29f416e. The videosAPI comment now says that status and content GETs have no body and are routed to a random endpoint, so async /v1/videos needs a single-replica pool or session affinity. /v1/videos/sync is unaffected.

@github-actions github-actions Bot added the needs-rebase PR has merge conflicts and needs a rebase label Oct 22, 2026
@github-actions

Copy link
Copy Markdown
Contributor

@0z5a This PR has merge conflicts with main. Please rebase on main and push to resolve them:

gh pr checkout 2937
git fetch origin main
git rebase origin/main
# resolve conflicts
git push --force-with-lease

0z5a added 2 commits October 26, 2026 13:17
The router no longer checks VideoGenerationRequest field bounds after
e98ff34, so the comments describing them were misleading. Document
that async status and content requests carry no body, so they are
routed to a random endpoint and need a single-replica pool or session
affinity.

Signed-off-by: 0z5a <[email protected]>
@0z5a
0z5a force-pushed the work/a2-video-parser branch from 29f416e to e0d4d14 Compare October 26, 2026 14:18
@github-actions github-actions Bot added needs-rebase PR has merge conflicts and needs a rebase and removed needs-rebase PR has merge conflicts and needs a rebase labels Oct 26, 2026
@github-actions

Copy link
Copy Markdown
Contributor

@0z5a This PR has merge conflicts with main. Please rebase on main and push to resolve them:

gh pr checkout 2937
git fetch origin main
git rebase origin/main
# resolve conflicts
git push --force-with-lease

switch name {
case reqcommon.FieldModel:
extractedBody.Model = string(value)
case "prompt":

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When rebasing onto main, carry #3721's case "negative_prompt": images.NegativePrompt = string(value) into this image-edits field switch, and keep its ParseRequest_ImagesEdits test row.

Why: main's parseImagesEditsRequest now parses negative_prompt for the image-edit token estimate (#3721). This PR replaces that function's body with the shared parseMultipartFields reader, and the merge conflict lands on exactly these cases, so resolving toward this branch silently drops the field and image-edit requests go back to being estimated without their negative prompt, with no compile error.

Comment on lines +59 to +65
// videosAPI is the OpenAI-compatible vLLM-Omni asynchronous video generation
// endpoint. Requests are multipart/form-data. Status and content requests
// (GET /v1/videos/{id} and /v1/videos/{id}/content) have no body, so they are
// routed to a random endpoint. With more than one replica, a poll can reach a
// pod that does not hold the job, so the pool needs a single replica or
// session affinity.
videosAPI = "videos"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move the async-polling caveat into parsers/README.md as a Drawback bullet under openai-parser, next to the Decisions one.

    *   **Drawback**: For async video generation, `POST /v1/videos`, the status and content requests (`GET /v1/videos/{id}`, `GET /v1/videos/{id}/content`) carry no body and are routed to a random endpoint, so a pool with more than one vLLM-Omni replica needs session affinity. `POST /v1/videos/sync` is unaffected.

Why: The caveat currently lives only in a Go constant's comment, and the parser README, where operators look up supported APIs, does not mention video at all. An operator who scales a vLLM-Omni pool past one replica gets intermittent 404s on job polls with nothing in the docs explaining why.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@0z5a Reminder: please add the async-polling caveat to parsers/README.md as a Drawback bullet under openai-parser (suggested text above). This is the last blocking item; the RewriteModelName revert is an optional nit. Everything else looks good, and the PR still merges cleanly and passes tests against current main.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 59c9f86. Added the Drawback bullet with your text under openai-parser in parsers/README.md, right after the Decisions one.

Comment on lines 212 to 219
func (p *OpenAIParser) RewriteModelName(payload fwkrh.MarshalablePayload, model string) (fwkrh.MarshalablePayload, error) {
m, ok := payload.(fwkrh.PayloadMap)
if !ok {
switch m := payload.(type) {
case fwkrh.PayloadMap:
m[reqcommon.FieldModel] = model
return m, nil
default:
return payload, nil
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: revert RewriteModelName to the original type assertion; this single-case type switch is left over from the removed MultipartPayload case.

m, ok := payload.(fwkrh.PayloadMap)
if !ok {
	return payload, nil
}
m[reqcommon.FieldModel] = model
return m, nil

Why: Video bodies are RawPayload now, so this function's behavior is unchanged from main and the diff here is churn in a hot-path rewriter that reviewers have to re-verify.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 59c9f86. RewriteModelName is back to the original type assertion (with reqcommon.FieldModel).

@ahg-g ahg-g self-assigned this Oct 30, 2026
Copilot AI balanced review requested due to automatic review settings October 31, 2026 12:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@github-actions github-actions Bot removed the needs-rebase PR has merge conflicts and needs a rebase label Oct 31, 2026
@0z5a
0z5a force-pushed the work/a2-video-parser branch from ef072ab to bb2b6be Compare October 31, 2026 12:01
if videos.Prompt == "" {
return nil, errors.New("invalid videos request: must have prompt field")
}
return &fwkrh.ParseResult{Body: extractedBody, SkipResponseProcessing: false}, nil

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Handle video/ responses alongside audio/ in OpenAIParser.ParseResponse, and add a video/mp4 case to TestOpenAIParser_ParseResponse.

if strings.HasPrefix(mediaType, "audio/") || strings.HasPrefix(mediaType, "video/") || mediaType == octetStreamType {

Why: parseVideosRequest sets SkipResponseProcessing: false, and POST /v1/videos/sync returns raw video/mp4 bytes -> ParseResponse misses the binary media-type branch and passes the MP4 body to extractUsage (json.Unmarshal) -> every synchronous video response logs a response-parsing error in HandleResponseBody and drops any x-vllm-omni-*-tokens usage headers.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. ParseResponse treats video/ like audio/ and reads usage from the response headers only. Added two video/mp4 cases to TestOpenAIParser_ParseResponse, with and without the x-vllm-omni-*-tokens headers. vLLM-Omni's /v1/videos/sync currently sends no token headers, so the case without headers is the one that matches today's server: it returns no usage and no error.

Comment on lines +653 to +658
case "fps":
fps, err := strconv.ParseFloat(string(value), 64)
if err != nil {
return fmt.Errorf("invalid videos fps field: %w", err)
}
videos.FPS = &fps

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reject NaN and Inf in parseVideosRequest's fps branch, and add "NaN" and "Inf" cases to TestOpenAIParser_ParseRequestVideosInvalidForms.

case "fps":
	fps, err := strconv.ParseFloat(string(value), 64)
	if err != nil || math.IsNaN(fps) || math.IsInf(fps, 0) {
		return fmt.Errorf("invalid videos fps field: %q", value)
	}
	videos.FPS = &fps

Why: strconv.ParseFloat accepts "NaN", "Inf", and "-Inf" without error -> non-finite floats enter VideoGenerationRequest.FPS -> downstream frame/cost arithmetic produces NaN/Inf and json.Marshal on VideoGenerationRequest fails with an unsupported-value error.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. The fps branch rejects NaN and Inf after ParseFloat. Added NaN, Inf and -Inf cases to TestOpenAIParser_ParseRequestVideosInvalidForms.

Comment on lines +630 to +644
func derefInt(v *int64) string {
if v == nil {
return "<nil>"
}
return strconv.FormatInt(*v, 10)
}

// derefFloat formats an optional float pointer for logging without
// dereferencing a nil, so String stays safe on a partially populated request.
func derefFloat(v *float64) string {
if v == nil {
return "<nil>"
}
return strconv.FormatFloat(*v, 'g', -1, 64)
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Return nilStr instead of "<nil>" in derefInt and derefFloat, and add TestVideoGenerationRequest_String in types_test.go.

Why: types.go already defines const nilStr = "<nil>" on line 36 (used on line 679 in VideoGenerationRequest.String()), so repeating raw "<nil>" literals duplicates the package constant, and VideoGenerationRequest.String() has no unit test alongside TestTextToSpeechRequest_String and TestAudioGenerateRequest_String.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. derefInt and derefFloat return nilStr, and TestVideoGenerationRequest_String covers a partially populated request and a nil receiver.

0z5a added 2 commits November 1, 2026 18:37
POST /v1/videos/sync returns raw video/mp4 bytes, which ParseResponse
passed to the JSON usage extractor. Treat video/ like audio/ and read
usage from the response headers. Reject NaN and Inf for the fps form
field, since strconv.ParseFloat accepts them. Use nilStr in derefInt and
derefFloat and add a VideoGenerationRequest.String test.

Signed-off-by: 0z5a <[email protected]>
Add the async /v1/videos polling caveat to the openai-parser section of
parsers/README.md, where operators look up supported APIs. Also restore
RewriteModelName's original type assertion; the type switch was left
over from the removed MultipartPayload case.

Signed-off-by: 0z5a <[email protected]>
@llm-d-router-release-notes
llm-d-router-release-notes Bot added this pull request to the merge queue Nov 2, 2026
Merged via the queue into llm-d:main with commit 63f2827 Nov 2, 2026
56 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/epp area/scheduling kind/feature Categorizes issue or PR as related to a new feature. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants