Skip to content

fix(pi): stop marking recovered turns as failed - #24805

Open
KyrosHendrix wants to merge 3 commits into
pingdotgg:mainfrom
KyrosHendrix:fix/pi-stale-message-failure
Open

KyrosHendrix wants to merge 3 commits into
pingdotgg:mainfrom
KyrosHendrix:fix/pi-stale-message-failure

Conversation

@KyrosHendrix

Copy link
Copy Markdown

Pi can finish a native continuation successfully while T3 persists a failed run and reports an earlier "This operation was aborted" message. The adapter retained an intermediate assistant error unless Pi emitted a specific retry-recovery event.

Track the latest assistant message failure separately from terminal failures. A successful assistant response clears the stale message error without clearing rejected prompts or exhausted retries. Explicit cancellation remains interrupted.

This is submitted under the small, focused obvious-bug exception. It corrects existing turn-completion bookkeeping, with no new feature, setting, contract, or database migration. The diff contains only the Pi adapter and regression tests for this defect.

Reproduction and evidence

The new orchestration regression replays this sanitized sequence through provider stdio, the real adapter, orchestration, persistence, and projections:

assistant message_end: error, "This operation was aborted"
agent_end: willRetry=false
agent_start
assistant message_end: stop
agent_settled: aborted=false

Original adapter: persisted run status = failed
Fixed adapter: run, attempt, provider turn = completed
               session = ready, lastError = null, no error item

With only the production adapter restored to the original base, the regression failed with expected [ 'failed' ] to deeply equal [ 'completed' ]. The same test passes with the fix. Adapter regressions also failed before implementation for recovery with and without compaction and for replacing an old error with the latest error.

Verification

Refreshed against upstream a96cf0818eff. Environment: Windows, Node 24.20.0, Vite+ 1.3.0.

vp test run packages/provider-pi/src/server/adapter.test.ts apps/server/src/orchestration-v2/testkit/PiReplayFixtures.integration.test.ts -t 'PiAdapterV2|PiRpc|recovered Pi message error'
vp run --parallel --filter t3 --filter @t3tools/provider-pi typecheck
vp fmt --check packages/provider-pi/src/server/adapter.ts packages/provider-pi/src/server/adapter.test.ts apps/server/src/orchestration-v2/testkit/PiReplayFixtures.integration.test.ts
vp lint packages/provider-pi/src/server/adapter.ts packages/provider-pi/src/server/adapter.test.ts apps/server/src/orchestration-v2/testkit/PiReplayFixtures.integration.test.ts
git diff --check
  • 97 focused tests passed: all 96 adapter tests and the new persisted orchestration regression. Eight existing replay cases were deselected. Controls cover genuine errors, unfinished continuation, non-retrying compaction, exhausted retries, rejected prompts, and cancellation.
  • Both scoped typechecks, formatting, lint, and diff checks passed. Existing unused layer and Duration warnings remain.
  • Additional earlier QA passed all nine replay cases with a temporary POSIX context matching their recorded subprocess arguments. That override for the existing cases is not committed. The new regression supplies its recording's POSIX context directly.

vp run check:changed was attempted but did not finish within 120 seconds; it printed the three-file change header without a step summary. It is not claimed as passing. A wider Pi-package run also hit Windows subprocess-fixture failures, including two status/discovery failures independently reproduced with the original adapter. These unrelated failures are not changed here. No live model-provider or browser verification ran. The original native abort trigger is not established, and historical failed turns are not rewritten.

Implemented and verified with openai-codex/gpt-6.1-sol in pi.

@github-actions github-actions Bot added vouch:unvouched Author is not yet trusted in the VOUCHED list. size:S 10-29 changed lines (additions + deletions). labels Oct 25, 2026
@macroscopeapp

macroscopeapp Bot commented Oct 25, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved at 5d4c49e

Macroscope's review found this PR approvable — This is a focused Pi adapter bug fix that separates transient message errors from terminal failures so recovered continuations persist as completed. It changes one production adapter and adds targeted unit and orchestration regression coverage without altering schemas, defaults, infrastructure, or static-analysis settings.

You can add or adjust custom eligibility rules. Learn more.

@coderabbitai

coderabbitai Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The Pi adapter now tracks assistant-message failures separately from other turn failures. Finalization considers these failures when determining terminal turn and compaction status. Tests cover successful recovery and cases where a message error remains the terminal failure.

Changes: Pi message failure recovery

Layer / File(s) Summary
Track and resolve message failures
packages/provider-pi/src/server/adapter.ts
The adapter records assistant-message errors separately, clears message failures after successful compaction retries and auto-retries, and considers them when determining terminal status.
Validate recovery and terminal status
packages/provider-pi/src/server/adapter.test.ts
Tests cover recovery after a message error and cases where that error, or a later retry error, remains the terminal failure.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Suggested reviewers: juliusmarminge

Merge Risk: Minimal · up to e77d6

This fix stops Pi turns that recovered after an intermediate error from being recorded as failed. Cancellation and unrecovered errors keep their existing behavior. No merge-blocking risk was identified.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title "fix(pi): stop marking recovered turns as failed" follows conventional commit format and matches the main change. The adapter now clears a stale assistant message error after a successful re…
Description check Passed The description covers the problem, the change, the scope justification, and detailed verification. It follows the template's Problem, Change, Scope, and Verification sections. It states the bug, expl…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title "fix(pi): stop marking recovered turns as failed" follows conventional commit format and matches the main change. The adapter now clears a stale assistant message error after a successful response, so recovered turns no longer persist as failed. The title is short, specific, and states the primary bug fix from the developer's perspective.

Full details: Description check

Explanation

The description covers the problem, the change, the scope justification, and detailed verification. It follows the template's Problem, Change, Scope, and Verification sections. It states the bug, explains the fix at the adapter level, cites the small obvious-bug exception with a link, and lists focused test commands, results, and unverified areas. It also names the agent and model used, as the template requests. The Problem section is folded into the opening paragraph rather than under a separate heading, which is a minor deviation.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 5d4c49e and e77d614.

Files selected for processing: 2

  • packages/provider-pi/src/server/adapter.test.ts
  • packages/provider-pi/src/server/adapter.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

Copy link
Copy Markdown
Collaborator

Note

Written by gpt-6.1-sol on behalf of Maria

Verified in a real dev environment

verified as-is at 5d4c49e243e099faaede49a7981dc1bd06cfcc73. reproduced the bug first on github/main at 32ecd71726 in /home/user/main, then repeated it on the exact pr head. one dev server ran at a time, with isolated state. no source changes or pushes.

used repository playwright with /usr/bin/google-chrome. created the project and threads through the web app. codex gpt-6.1-sol completed a baseline turn on both revisions. the changed path used real pi 1.4.0 with cliproxyapi/gpt-6.1-sol through this vm's proxy using its chat-completions api.

reproduction: disable pi's automatic retries, start a long response, click "steer message" with Reply exactly RECOVERY_OK. Do not use tools or change files., then disconnect only pi's active tcp socket with sudo -n ss -K dst 127.0.0.1 dport = 28317 sport = <pi-client-port>. this induces a real connection failure without replacing the provider, injecting events, or using replay data. both native pi sessions recorded stopReason=error with Connection error., followed by stopReason=stop with RECOVERY_OK.

read-only sqlite inspection after the ui completed:

                         main                 pr head
assistant response       RECOVERY_OK          RECOVERY_OK
run / attempt / turn     failed               completed
provider session         error                ready
lastError                Connection error.    null
persisted error items    1                    0

neighboring checks on the pr: disconnecting pi without a steer still failed; "stop generation" persisted interrupted; another message after stop completed with AFTER_STOP_OK. reloading preserved the recovered response and clean state.

blacksmith vp run check:changed a96cf0818eff4a113c3ff52ae702683e51a5803b passed format, lint, knip, graphql, five affected typechecks, and all 105 tests in the two affected test files. the explicit base was needed because the worker starts with shallow history. two existing unused-variable lint warnings remain.

screenshots use matching prompts and response data. the recording shows the steer and successful continuation; loading screens were removed. uploaded images, video playback, and seeking were checked in chrome.

main: successful continuation incorrectly persists a connection failure

pr head: the same continuation completes without a stale error

pi steer and successful recovery in the running pr app

Not verified

  • the original "This operation was aborted" trigger was not reproduced. the live reproduction exercised the same stale-message-error bookkeeping with a controlled connection failure.
  • pi's responses api could not complete through this proxy, returning empty-response errors; live pi verification used chat completions.
  • native ios/android, electron, macos/windows, remote relay/tunnel, and live compaction/exhausted-retry recovery were not exercised on this linux vm. focused existing tests cover the latter terminal semantics.

@KyrosHendrix

KyrosHendrix commented Nov 11, 2026 •

Copy link
Copy Markdown
Author

Following up on this fix. I am still encountering the reported bug. Could a maintainer take another look?

A fresh source review against main at 7e565c510d6dfaef2e1fd97bf9d89a97ce9cece7 confirms that the stale-error path is still present: an assistant message_end with stopReason: error sets turn.failure, and a later successful assistant message does not clear it. Without an explicit successful retry or retrying compaction event, finalization still marks the recovered turn failed.

Re-reviewed PR head 5d4c49e243e099faaede49a7981dc1bd06cfcc73 and found no new correctness issues. The patch clears recovered assistant-message failures while retaining terminal failures and cancellation semantics.

Fresh verification on the unchanged PR head:

  • PASS: pnpm exec vp test run packages/provider-pi/src/server/adapter.test.ts apps/server/src/orchestration-v2/testkit/PiReplayFixtures.integration.test.ts -t 'PiAdapterV2|PiRpc|recovered Pi message error', exit 0. Both test files passed, 97 tests passed and 8 were skipped by the name filter.
  • PASS: git diff --check, exit 0.

This follow-up includes source review and focused tests, not a new live-provider or UI reproduction. The earlier live verification in #24805 (comment) remains relevant.

@juliusmarminge, since you've worked on Pi's compaction and steering behavior, could you take a look when you have time?

@juliusmarminge juliusmarminge added the takeover:in-progress A takeover lead is working on this PR label Nov 14, 2026
juliusmarminge and others added 2 commits November 14, 2026 07:23
Drop the replay-fixture case that repeated the adapter test one layer up,
and the table rows that a sibling row or unchanged code already covers.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:S 10-29 changed lines (additions + deletions). takeover:in-progress A takeover lead is working on this PR vouch:unvouched Author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants