You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Triplet view collapses requests without valid prompt token IDs #572
The format=triplet event view currently groups every model_request by prompt_token_ids and keeps the last event for each key. When those IDs are absent or empty, every request gets the same empty key, so unrelated calls in one rollout are silently collapsed.
Malformed values can be worse: a nested list reaches tuple(prompt_token_ids) and raises TypeError, turning the triplet query into a 500 response.
This is reachable through the event POST endpoint, whose event data is intentionally open-ended, and through OpenAI-compatible responses that do not echo prompt token IDs. _trim_model_request also defaults missing IDs to an empty list.
Minimal reproduction
Append two model_request events to one rollout with different non-empty response token IDs.
Give both events prompt_token_ids: [] (or omit the field).
Query /api/rollouts/{id}/events?format=triplet.
Only the second request is returned. Using prompt_token_ids: [[1]] instead raises TypeError: unhashable type: 'list'.
Expected behavior
Prompt-based last-wins deduplication should remain unchanged when IDs are a valid, non-empty list of integers. When no valid key exists, the events should be retained in insertion order because the value cannot establish that they are duplicates.
I have a focused fix with server and VERL multimodal-alignment regression coverage ready and will link the PR here.
Thanks — I've simplified #573 along those lines, keeping the original last-index map and removing the two helpers and superseded-index tracking.
One detail: the guard needs to retain the request's index before continue, otherwise the final inclusion filter drops requests with missing/empty IDs. I also kept the integer-list check: [[1]] is non-empty but still raises when used as a key. The VERL image-alignment path follows the same rule.
The full CPU test suite passes (92 tests), along with Ruff, Pyright, pre-commit and the package build. The 16 invalid-ID regressions also fail on unchanged upstream, so they exercise the original failure modes.
Activity
hzy46 commentedon Aug 28, 2026
Thanks for report this issue.
For the PR, can the code be simplifed to just add the following sentences?
rioyu123 commentedon Aug 28, 2026
Thanks — I've simplified #573 along those lines, keeping the original last-index map and removing the two helpers and superseded-index tracking.
One detail: the guard needs to retain the request's index before
continue, otherwise the final inclusion filter drops requests with missing/empty IDs. I also kept the integer-list check:[[1]]is non-empty but still raises when used as a key. The VERL image-alignment path follows the same rule.The full CPU test suite passes (92 tests), along with Ruff, Pyright, pre-commit and the package build. The 16 invalid-ID regressions also fail on unchanged upstream, so they exercise the original failure modes.
hzy46 commentedon Aug 28, 2026
Merged. Thank you for this contribution.