Repository navigation
fix(stream): expired model lock masks account lock, and dropped generated images (#3516, #3521) - #3879
fix(stream): expired model lock masks account lock, and dropped generated images (#3516, #3521)#3879ntdatt812 wants to merge 3 commits into
Conversation
…count lock isModelLockActive picked the per-model key on the truthiness of the string: const expiry = connection[key] || connection[MODEL_LOCK_ALL]; so a stale `modelLock_<model>` won over a still-active `modelLock___all`. The connection then read as free for exactly the model that had just failed, while still reading as locked for every other model: modelLock_claude-fable-5 = <1 minute ago> modelLock___all = <next month> isModelLockActive(conn, "claude-fable-5") -> false (expected true) isModelLockActive(conn, "some-other-model") -> true getEarliestModelLockUntil(conn) -> next month That shape is reachable and already documented by the suite: a plain GitHub 402 sets a two-minute per-model lock, and monthly exhaustion sets an account-wide one until the next UTC month. Two minutes later the first has expired and the second has not. Nothing clears the stale key either — the lazy cleanup only runs after a successful request, which an exhausted account never gets. Judge each key on its own expiry, which is the rule the sibling getEarliestModelLockUntil already applies when it skips expired entries. An unparseable timestamp is treated as no lock, so a malformed value cannot pin a connection forever.
`hasValuableContent` gates every chunk on the streaming path
(open-sse/utils/stream.js:315). Its OpenAI branch listed content,
reasoning_content, tool_calls, finish_reason and role — but not images.
A generated image arrives on its own chunk with nothing else in the delta, so
every one of them was discarded:
delta={"role":"assistant"} kept
delta={"images":[{"type":"image_url","image_url":{...}}]} DROPPED
The translator emits exactly that shape (translator/response/gemini-to-openai.js:105)
and the repo's golden snapshot locks it under "image output (inlineData ->
delta.images)" — so the filter was throwing away a payload the project had
already committed to producing.
Streaming an image-generating model through /v1/chat/completions therefore
returned a well-formed but empty assistant message. Non-streaming is unaffected.
An empty `images` array stays filtered, so a stub chunk is not promoted to
content.
Review follow-up on this PR. The per-key expiry treatment asked for was already there -- `getEarliestModelLockUntil` skips expired entries -- but the two helpers diverge on something else: scope. `isModelLockActive(c, model)` is model-scoped. `getEarliestModelLockUntil(c)` is connection-wide, so on a connection locked for two models it returns the unrelated model's expiry. Both retry-timing call sites in auth.js used it, so a connection locked for `claude-fable-5` until 10:00 while holding a lock for another model until 09:30 advised a retry at 09:30. The retry hits the same lock and fails identically: one wasted upstream request per caller, surfaced as a second rate-limit error. The debug line beside it printed `modelLocked(<model>) until <other model's expiry>`. `getModelLockUntil(connection, model)` is the time-valued twin of `isModelLockActive`: same two keys, same `getModelLockKey` normalisation so a null or absent model resolves account-wide in both, and the **later** of the two gating expiries, because the connection frees up only once both have passed. `getEarliestModelLockUntil` is left alone. It is the right question for a per-connection cooldown badge, and widening it on argument arity would reintroduce the undefined-versus-null divergence this file already had one of. Also adds the stream-end image test asked for. The existing one calls `hasValuableContent` directly, which proves the predicate and nothing about the stream consulting it; these drive a real Gemini->OpenAI transform with the image part as the last thing the provider sends, where no later chunk can carry it instead. Mutation-checked: min instead of max, scanning every `modelLock_*` key, and dropping the account-wide gate each turn a test red. Reverting the `delta.images` line turns all four stream tests red.
3fb3ba6 to
b75fcdd
Compare
|
Both addressed in The images test — exactly as asked
Reverting the The lock helpers — the divergence is scope, not expiryI went to add the per-key expiry treatment and found it is already there: But you were pointing at something real one level over. const lockedConns = connections.filter(c => isModelLockActive(c, model));
const expiries = lockedConns.map(c => getEarliestModelLockUntil(c)).filter(Boolean);A connection locked for
I did not widen 5 new tests in Mutation-checked, since the subtle wrong version here passes a naive test:
Verification
Also re-checked against current master before claiming the commits are still needed: The three commits remain independent — the new one is |
Two stream-path fixes that were sitting in separate PRs (#3516, #3521), rebased onto current
master. Independent commits; either can be dropped.fix(fallback)(#3516) — an expired per-model lock masked an active account lock. The per-model entry was checked first and, once its TTL had passed, the code returned "not locked" without consulting the account-level lock that was still live, so requests were sent to an account the router had already taken out of rotation.fix(stream)(#3521) — generated images were dropped from OpenAI-format streams. The delta handler forwardedcontentand reasoning fields and ignored the image parts, so a model that returned an image mid-stream produced text-only output with no error.Not included: #3542
I tried to bundle #3542 (record the turn when a client hangs up mid-stream) and it no longer applies.
masterhas since grown its ownfinalizeStream()inopen-sse/utils/stream.js, called fromtransformon terminal events and fromflush, which overlaps that PR'sfinishStream(). The remaining gap is real — there is still nocancelcallback on the transformer, so a client that disconnects before any terminal event still records nothing — but closing it now means rewriting against the new structure rather than replaying the old patch. I will send that separately rather than force a stale diff through here.Verification
Run with vitest 3 and the repo config — both parts matter:
Without
--configthe@/alias does not resolve (Failed to load url @/lib/dataDir.js). And vitest 2.x cannot run anything that reachesopen-sse/translator/index.js: it throwsTypeError: register is not a functionat the top-levelregister(...)calls, becauseindex.jsdeliberately relies on function-declaration hoisting to survive the import cycle while Vite 5's SSR transform turns the named import into a call-time property access. I confirmed that failure on cleanmasterand onmasteras of 15 Aug, so it is not caused by these changes.Every fix was checked by reverting it and confirming which test dies. No claim here rests on a test that passes for another reason.