Skip to content

fix(error-status): parse "Error: <status>" with no colon after the status - #64

Merged
eiei114 merged 3 commits into
eiei114:mainfrom
tc3oliver:fix/error-prefix-status-without-colon
Sep 21, 2026
Merged

eiei114 merged 3 commits into
eiei114:mainfrom
tc3oliver:fix/error-prefix-status-without-colon

Conversation

@tc3oliver

@tc3oliver tc3oliver commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • GitHub Copilot reports quota exhaustion as Error: 429 quota exceeded, and parseStatusFromErrorMessage returns undefined for it. turn_end then returns early, no rule can match, and the fallback never fires.
  • The message falls between two existing patterns. The leading-bare-status pattern accepts an Error: prefix but requires a colon after the status (Error: 401: {...}, added in fix: parse leading bare status codes in error messages #55). The error <status> pattern requires whitespace directly after error, so the colon in Error: defeats it.
  • Adds an anchored pattern that requires the literal word error, and adds quota to the trailing-noun list so a bare 429 quota exceeded parses too.

Observed in a real session (Pi 0.85.1, github-copilot provider):

 hi

 Error: 429 quota exceeded

/model-fallback:status showed Rules: 3, Config warnings: none, and state.json was never created — persistFailure() was never reached.

Before / after:

"error 429"                  -> 429        (both)
"Error: 429 quota exceeded"  -> undefined  ->  429
"429 quota exceeded"         -> undefined  ->  429

Not broadening the match

The prefix requirement is deliberate. Dropping it would also match prose the existing tests require to stay unmatched:

parseStatusFromErrorMessage("429 tokens remaining in context window") // undefined
parseStatusFromErrorMessage("processed 404 items before retry")       // undefined

The quota noun relies on adjacency for the same reason, so 429 tokens left before the quota resets still does not match. Both cases are added as regression assertions.

Note on #58

#58 also touches lib/error-status.ts (adding reasons). The two changes are independent — this one only widens status extraction — but whichever lands second will need a trivial rebase.

Testing

  • npm run ci — 39 tests pass, up from 37 on main
  • pi -e .

On pi -e .: the fix was validated by applying it to an installed 0.4.0 and re-running parseStatusFromErrorMessage against the verbatim error string from the session above, which now yields 429. A full interactive confirmation needs another Copilot quota exhaustion to reproduce, so I have not checked that box. Happy to follow up with it once the allowance next runs out.

Checklist

  • README updated if user-facing behavior changed — no user-facing surface changed, so no README change
  • CHANGELOG updated if needed — added under Unreleased
  • Package contents checked with npm pack --dry-run — unchanged, no docs/ files added or renamed

Summary by CodeRabbit

  • New Features

    • Added optional failure-reason matching for fallback rules.
    • Fallbacks can now cascade across multiple models without revisiting models used in the current run.
    • Rules no longer fall back to the model that produced the failure.
  • Bug Fixes

    • Improved detection of provider errors such as Error: 429 quota exceeded.
    • Added recognition of context-length failure reasons for more precise fallback behavior.
  • Documentation

    • Updated configuration and usage guidance for reason-based rules and cascading fallbacks.

…atus

GitHub Copilot reports quota exhaustion as:

    Error: 429 quota exceeded

That message fell between two existing patterns. The leading-bare-status
pattern accepts an "Error:" prefix but requires a colon AFTER the status,
matching "Error: 401: {...}". The "error <status>" pattern requires
whitespace directly after "error", so the colon in "Error:" defeats it.
With no status parsed, turn_end returns early and no rule can fire, so
Copilot quota exhaustion never triggered a fallback.

The new pattern is anchored and requires the literal word "error". That
matters: dropping the prefix requirement would also match prose such as
"429 tokens remaining in context window", which the existing tests
require to stay unmatched.

Also recognises a bare status followed by a quota noun, so
"429 quota exceeded" parses on its own. This relies on adjacency, so
"429 tokens left before the quota resets" still does not match.

npm run ci passes: 39 tests, up from 37.
@coderabbitai

coderabbitai Bot commented Sep 18, 2026 •

Copy link
Copy Markdown

Review Change StackReview Change Stack

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: c15f7fe0-d07c-433c-a7e1-f1b1f4a06a75

📥 Commits

Reviewing files that changed from the base of the PR and between 4d49ab3 and a0f5cad.

📒 Files selected for processing (9)
  • CHANGELOG.md
  • README.md
  • docs/usage.md
  • extensions/index.ts
  • lib/config.ts
  • lib/error-status.ts
  • tests/auto-retry.test.mjs
  • tests/config.test.mjs
  • tests/error-status.test.mjs
 _____________________________________
< CORS error? I'll cross that bridge. >
 -------------------------------------
  \
   \   \
        \ /\
        ( )
      .( o ).
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Belphemur and others added 2 commits September 21, 2026 14:59
…iei114#58)

* feat(rules): match failure reasons such as context_length_exceeded

Add an optional `reasons` field to fallback rules. When set, a rule only
fires if the failure reason parsed from the provider error message is
listed. Reasons come from quoted machine codes (for example
"provider_error_code":"context_length_exceeded") or prose such as
"exceeds this model's context length". This lets status 400 trigger a
fallback for context-length overflow while other 400s stay errors.

Also skip rules whose fallback target equals the failing model, so a
broad provider rule can safely target a model inside that provider.

Reason sets participate in shadowing analysis: unrestricted earlier
rules shadow later reason-scoped rules, but rules with disjoint reason
sets on the same scope are not reported as duplicates.

* feat(fallback): cascade through chains while blocking circular loops

Fallback targets can now fail and cascade onward via other matching rules
(e.g. free -> kimi-k2.6 -> glm-5.3-flash) instead of surfacing the error.
A per-run visited set blocks revisiting any model already used, so
circular rules cannot ping-pong; re-applying the preselected fallback for
an error attributed to the original model stays allowed (idempotent).
@eiei114
eiei114 merged commit 640f09e into eiei114:main Sep 21, 2026
1 of 2 checks passed
@eiei114

eiei114 commented Sep 21, 2026

Copy link
Copy Markdown
Owner

Thanks @tc3oliver — merged as 640f09e.

The diagnosis was the useful part here: the two existing patterns each required something the other shape did not have, and keeping the error prefix mandatory so 429 tokens remaining in context window stays unmatched was the right call. The regression assertions you added for the prose cases are what made this safe to take straight in.

One note for next time: you predicted the rebase against #58 in the PR body, and it landed exactly as you said — a single CHANGELOG conflict, code auto-merged. I resolved it on your branch and pushed the merge commit, so the credit and history stay with your PR.

Nice work."

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants