Skip to content

fix(openai): surface chat-completions refusal text#102342

Closed
arkyu2077 wants to merge 1 commit into
openclaw:mainfrom
arkyu2077:fix/issue-102321
Closed

fix(openai): surface chat-completions refusal text#102342
arkyu2077 wants to merge 1 commit into
openclaw:mainfrom
arkyu2077:fix/issue-102321

Conversation

@arkyu2077

Copy link
Copy Markdown
Contributor

Summary

  • surface choice.delta.refusal in processOpenAICompletionsStream
  • also surface aggregated choice.message.refusal payloads from OpenAI-compatible chat-completions APIs
  • add regression coverage for both stream chunk and aggregated message refusal-only responses

Testing

  • pnpm vitest run src/agents/openai-transport-stream.test.ts

Fixes #102321

@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: S triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. labels Jul 9, 2026
@clawsweeper

clawsweeper Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed July 8, 2026, 9:51 PM ET / 01:51 UTC.

Summary
The PR adds refusal-field handling to processOpenAICompletionsStream plus agent-transport regression tests for streamed and aggregated refusal-only chat-completions responses.

PR surface: Source +3, Tests +80. Total +83 across 2 files.

Reproducibility: yes. Source inspection gives a high-confidence path: feed either OpenAI-completions parser a refusal-only delta or message and current main reads content, reasoning, and tool calls but not refusal.

Review metrics: none identified.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #102321
Summary: This PR is a partial candidate fix for the exact chat-completions refusal-drop issue; other related refusal items cover different API paths or fallback policy.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🧂 unranked krab
Patch quality: 🦪 silver shellfish
Result: blocked until real behavior proof from a real setup is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P1] Fix the aggregated refusal fixture so check-test-types passes.
  • Apply the same refusal handling and focused tests to packages/ai/src/providers/openai-completions.ts.
  • [P1] Add redacted real runtime proof from an OpenAI-compatible turn showing refusal text rendered as visible assistant content.

Proof guidance:

  • [P1] Needs real behavior proof before merge: The PR body lists only a Vitest command, so it still needs redacted real behavior proof such as terminal output, logs, copied live output, or a recording from an OpenAI-compatible runtime turn. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

  • [P1] The branch is not mergeable as submitted because check-test-types fails on the new aggregated refusal fixture.
  • [P1] Merging this branch would only repair the agent transport path while the shared packages/ai OpenAI-completions parser can still drop refusal-only replies.
  • [P1] The PR body lists only a Vitest command, so there is no redacted real runtime proof from an OpenAI-compatible turn yet.

Maintainer options:

  1. Decide the mitigation before merge
    Use one complete refusal-to-visible-text fix across both OpenAI-completions assemblers, with green type checks, focused regression coverage, and redacted real runtime proof while leaving fallback/error policy to separate work.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

  • [P1] Contributor or maintainer follow-up is needed because the branch has a typecheck failure, an unfixed live sibling parser, and missing real behavior proof; automation should not repair this while proof remains contributor-owned.

Security
Cleared: The diff parses one additional provider response field and adds tests; it does not touch secrets, dependencies, workflows, package metadata, or code execution paths.

Review findings

  • [P1] Fix the aggregated refusal fixture type — src/agents/openai-transport-stream.test.ts:2787-2793
  • [P2] Mirror refusal handling in the shared completions provider — src/agents/openai-transport-stream.ts:3169-3170
Review details

Best possible solution:

Use one complete refusal-to-visible-text fix across both OpenAI-completions assemblers, with green type checks, focused regression coverage, and redacted real runtime proof while leaving fallback/error policy to separate work.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection gives a high-confidence path: feed either OpenAI-completions parser a refusal-only delta or message and current main reads content, reasoning, and tool calls but not refusal.

Is this the best way to solve the issue?

No. The narrow parser-parity direction is right, but this branch is incomplete until it typechecks and covers both live completions assemblers with proof.

Full review comments:

  • [P1] Fix the aggregated refusal fixture type — src/agents/openai-transport-stream.test.ts:2787-2793
    The current check-test-types job fails here because processOpenAICompletionsStream accepts AsyncIterable<ChatCompletionChunk>, but this fixture yields object: "chat.completion" with a message choice and no required delta. Please make the aggregated shape an explicit tolerated cast/helper or otherwise keep the fixture assignable so type checks pass.
    Confidence: 0.94
  • [P2] Mirror refusal handling in the shared completions provider — src/agents/openai-transport-stream.ts:3169-3170
    This fixes only the agent transport parser, but packages/ai/src/providers/openai-completions.ts is a live built-in openai-completions assembler and still handles content/tool calls without reading choice.delta.refusal. Users routed through that shared provider path can still get an empty assistant turn, so the sibling parser and package-level regression coverage need the same repair.
    Confidence: 0.9

Overall correctness: patch is incorrect
Overall confidence: 0.92

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 6630138598b3.

Label changes

Label changes:

  • add P2: This PR addresses a normal-priority provider-output bug with limited blast radius, but the current patch still needs correctness and proof follow-up.
  • add rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🦪 silver shellfish.
  • add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body lists only a Vitest command, so it still needs redacted real behavior proof such as terminal output, logs, copied live output, or a recording from an OpenAI-compatible runtime turn. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Label justifications:

  • P2: This PR addresses a normal-priority provider-output bug with limited blast radius, but the current patch still needs correctness and proof follow-up.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🦪 silver shellfish.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body lists only a Vitest command, so it still needs redacted real behavior proof such as terminal output, logs, copied live output, or a recording from an OpenAI-compatible runtime turn. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed

PR surface:

Source +3, Tests +80. Total +83 across 2 files.

View PR surface stats
Area Files Added Removed Net
Source 1 3 0 +3
Tests 1 80 0 +80
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 2 83 0 +83

What I checked:

Likely related people:

  • vincentkoc: Current blame on both affected parser loops resolves to a recent merged carrier commit, and earlier history shows provider transport seam work in the same agent transport surface. (role: recent area contributor; confidence: high; commits: eb5d8f85c458, 4798e125f40c; files: src/agents/openai-transport-stream.ts, packages/ai/src/providers/openai-completions.ts)
  • steipete: Recent merged history touches the shared packages/ai OpenAI-completions provider and built-in registration boundary. (role: adjacent package-runtime contributor; confidence: medium; commits: 4bf70be01a21; files: packages/ai/src/providers/openai-completions.ts, packages/ai/src/providers/register-builtins.ts)
  • balajisiva: Authored a nearby merged OpenAI-compatible completions silent-drop repair for vLLM reasoning and empty tool-call response parsing. (role: adjacent bug-fix author; confidence: medium; commits: efab9763dc87; files: src/agents/openai-transport-stream.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. labels Jul 9, 2026
@vincentkoc vincentkoc self-assigned this Jul 9, 2026
@vincentkoc

Copy link
Copy Markdown
Member

Thanks for the focused refusal handling and regression coverage. This is now superseded by #102344, landed on main as 3fd5251.

The landed fix covers the same agent choice.delta.refusal and compatible choice.message.refusal shapes, and also fixes the sibling registered packages/ai chat-completions assembler with matching tests. The canonical issue #102321 closed with that merge, so I’m closing this duplicate.

@vincentkoc vincentkoc closed this Jul 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling P2 Normal backlog priority with limited blast radius. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. size: S status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

openai-completions: chat-completions refusal field dropped, assistant turn resolves to empty content

2 participants