Skip to content

feat(config): object-style fallback_models with per-model settings#2622

Merged
code-yeongyu merged 3 commits intocode-yeongyu:devfrom
RaviTharuma:feat/object-style-fallback-models
Mar 25, 2026
Merged

feat(config): object-style fallback_models with per-model settings#2622
code-yeongyu merged 3 commits intocode-yeongyu:devfrom
RaviTharuma:feat/object-style-fallback-models

Conversation

@RaviTharuma
Copy link
Copy Markdown
Contributor

@RaviTharuma RaviTharuma commented Mar 16, 2026

PR Stack (1/3) — merge in order:

  1. feat(config): object-style fallback_models with per-model settings #2622 ← this PR (base: dev)
  2. feat(settings): add model settings compatibility resolver #2643 — model settings compatibility resolver (based on this PR)
  3. refactor: deduplicate DelegatedModelConfig into shared module #2674 — deduplicate DelegatedModelConfig (based on feat(settings): add model settings compatibility resolver #2643)

Summary

This PR makes fallback_models actually usable across different model families.

The real problem was that a fallback chain is not just a list of model IDs. Different fallback models often need different settings to work properly:

  • Anthropic models may need a different variant
  • OpenAI models may need a different reasoningEffort
  • other providers may need their own temperature, top_p, maxTokens, or thinking settings

Without per-model settings, fallback was fragile:

  • some fallback models would fail outright because they inherited settings the fallback model does not support
  • others would technically run, but perform much worse because they were missing the settings they actually need

A concrete example is falling back from Opus with variant: "max" to Sonnet: the fallback inherits "max", but Sonnet does not support that level, so the fallback fails instead of rescuing the task.

This PR fixes that by letting each fallback entry carry the settings that belong to that specific model, so the fallback chain behaves like a real fallback system instead of a best-effort model swap.

What changed

  • Adds object-style entries to fallback_models, so each fallback model can define its own:
    • variant
    • reasoningEffort
    • temperature
    • top_p
    • maxTokens
    • thinking
  • Keeps string entries working exactly as before
  • Promotes the chosen fallback entry's settings when that fallback becomes the selected execution model
  • Aligns delegated/background prompt payloads with the current OpenCode SDK contract:
    • prompt bodies now only send supported fields: { model: { providerID, modelID }, variant }
    • model-specific tuning is applied via per-session chat.params
  • Fixes fuzzy fallback matching so promoted settings still apply when model resolution returns an extended model ID such as:
    • gpt-5.4 -> gpt-5.4-preview
    • gpt-5.4 -> gpt-5.4o
  • Prefers the most specific matching fallback entry when multiple configured models share a prefix

Example config

"fallback_models": [
  { "model": "anthropic/claude-sonnet-4-6", "variant": "high", "reasoningEffort": "high" },
  { "model": "openai/gpt-5.4", "reasoningEffort": "xhigh" },
  "chutes/kimi-k2.5",
  { "model": "chutes/glm-5", "temperature": 0.7 },
  "google/gemini-3-flash"
]

Why this matters

This is important because fallback quality depends on more than swapping the model name.

If the fallback model does not receive the settings it expects, one of two bad things happens:

  1. the fallback fails because the inherited settings are invalid for that model
  2. the fallback succeeds, but behaves badly because it is running with the wrong tuning

The point of a fallback chain is reliability. This PR makes fallback entries model-aware, so each fallback can run with the settings that actually fit that model.

Test plan

  • Added regression coverage for object-style fallback parsing and promotion
  • Added SDK-compatibility regression tests for delegated/background prompt payloads
  • Added matching coverage for exact, fuzzy-extension, and shared-prefix fallback cases
  • bun run typecheck
  • bun test -> 4150 pass, 0 fail

Closes #2621


Summary by cubic

Adds object-style fallback_models with per-model settings and promotes them only when a real fallback is selected across delegate tasks, subagents, background agents, and runtime-fallback (closes #2621). Prompt payloads now send only provider/model IDs and a top-level variant; per-model tuning is stored in session-scoped chat.params, applied on prompts, restored on resume, and cleared on session.deleted.

  • New Features

    • Support object-style entries in fallback_models with variant, reasoningEffort (adds minimal), temperature, top_p, maxTokens, thinking; mixed arrays allowed; Zod-validated schema.
    • Fallback chain accepts mixed entries; most-specific prefix match via findMostSpecificFallbackEntry; fuzzy model resolution returns the matched entry so per-model settings carry through; flattenToFallbackModelStrings() feeds runtime-fallback.
    • Chosen fallback settings are promoted into delegated model config and persisted as per-session prompt params; applied in sync/background prompts, restored on resume, and cleaned up on session.deleted.
    • Utilities: normalizeFallbackModels, flattenToFallbackModelStrings(), session prompt params store, and shared KNOWN_VARIANTS.
  • Bug Fixes

    • Gate per-model setting promotion to actual fallback matches in delegate-task flows.
    • Inline-variant handling: avoid double suffixes; explicit object variant overrides inline; preserve inline when not overridden; recognize minimal.

Written for commit fb08553. Summary will update on new commits.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d873abb88a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 8 files

Confidence score: 3/5

  • There is a concrete regression risk in src/hooks/runtime-fallback/auto-retry.ts: object-style per-model settings can be lost during auto-retry because config is flattened to a string and only provider/model(/variant) is reconstructed.
  • Given the severity (6/10) and high confidence (9/10), this is more than a minor edge case and could cause user-facing behavior changes when retries occur with custom model options.
  • Pay close attention to src/hooks/runtime-fallback/auto-retry.ts - ensure auto-retry preserves full per-model object settings instead of dropping non-provider/model fields.
Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/hooks/runtime-fallback/auto-retry.ts">

<violation number="1" location="src/hooks/runtime-fallback/auto-retry.ts:118">
P2: Auto-retry fallback drops object-style per-model settings by flattening to string and reconstructing only provider/model(/variant).</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 1 file (changes from recent commits).

Requires human review: Auto-approval blocked by 1 unresolved issue from previous reviews.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2d82727956

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 2 files (changes from recent commits).

Requires human review: Auto-approval blocked by 1 unresolved issue from previous reviews.

@RaviTharuma RaviTharuma force-pushed the feat/object-style-fallback-models branch from 2c3092e to 69e0f70 Compare March 16, 2026 14:20
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 69e0f70910

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 3 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/tools/delegate-task/subagent-resolver.ts">

<violation number="1" location="src/tools/delegate-task/subagent-resolver.ts:120">
P2: Object-style fallback model settings are discarded during static fallback resolution because fallback objects are flattened to strings before selection.</violation>
</file>

<file name="src/tools/delegate-task/category-resolver.ts">

<violation number="1" location="src/tools/delegate-task/category-resolver.ts:105">
P2: Flattening fallback objects to strings drops per-model settings when a fallback model is promoted to the initial execution model.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

@RaviTharuma
Copy link
Copy Markdown
Contributor Author

Follow-up: the newly reported delegate-task issues are fixed in 31ba06c and I resolved those review threads. The only remaining unresolved thread is the earlier runtime-fallback SDK limitation one — that path still cannot carry object-level per-prompt settings because the current OpenCode prompt API only accepts { model, variant } there. If upstream support for per-prompt overrides lands later, I can wire that path up too.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 31ba06ce66

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4 issues found across 17 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/shared/model-resolver.ts">

<violation number="1" location="src/shared/model-resolver.ts:96">
P1: Custom agent: **Opencode Compatibility**

Add `"minimal"` to the list of supported inline variants, as it is a valid variant supported by Opencode (e.g., for OpenAI models).</violation>
</file>

<file name="src/tools/delegate-task/sync-prompt-sender.ts">

<violation number="1" location="src/tools/delegate-task/sync-prompt-sender.ts:80">
P2: `categoryModel.maxTokens` and `categoryModel.thinking` are added and populated upstream but never included in the prompt payload, so these fallback model settings are silently ignored at runtime.</violation>
</file>

<file name="src/features/background-agent/manager.ts">

<violation number="1" location="src/features/background-agent/manager.ts:488">
P2: `resume()` drops new per-model settings (`temperature`, `top_p`, `reasoningEffort`), so resumed tasks can run with different model behavior than initial launch.</violation>
</file>

<file name="src/tools/delegate-task/category-resolver.ts">

<violation number="1" location="src/tools/delegate-task/category-resolver.ts:23">
P2: `getFallbackModelConfig` uses exact string equality to match `actualModel` against fallback chain entries, but the model resolver can return a fuzzy match (e.g., `openai/gpt-5.4-preview` when the user configured `openai/gpt-5.4`). In that case the lookup misses and all per-entry settings (`reasoningEffort`, `temperature`, `top_p`, etc.) are silently dropped. Consider using a prefix or `startsWith` match on the model ID portion to handle resolved model names that extend the configured base name.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 809b1a2750

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

5 issues found across 14 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/features/background-agent/manager.ts">

<violation number="1" location="src/features/background-agent/manager.ts:768">
P1: Custom agent: **Opencode Compatibility**

The Opencode SDK and `PromptInput` schema do not support `temperature` or `top_p` in the `model` object.</violation>

<violation number="2" location="src/features/background-agent/manager.ts:787">
P1: Custom agent: **Opencode Compatibility**

The Opencode SDK and `PromptInput` schema do not support passing an `options` object (such as `reasoningEffort`, `thinking`, or `maxTokens`) in a prompt request.</violation>
</file>

<file name="src/tools/delegate-task/sync-prompt-sender.ts">

<violation number="1" location="src/tools/delegate-task/sync-prompt-sender.ts:87">
P1: Custom agent: **Opencode Compatibility**

The `options` object is not supported by the Opencode SDK's session prompt payload.</violation>
</file>

<file name="src/tools/delegate-task/subagent-resolver.ts">

<violation number="1" location="src/tools/delegate-task/subagent-resolver.ts:37">
P2: Combined exact/prefix matching inside a single `.find()` lets broad base-model entries shadow later exact specific-model configs.</violation>
</file>

<file name="src/tools/delegate-task/category-resolver.ts">

<violation number="1" location="src/tools/delegate-task/category-resolver.ts:25">
P2: The `startsWith(`${configuredModel}-`)` check is stricter than the upstream `fuzzyMatchModel` substring matching. If fuzzy resolution produces a model ID that extends the configured prefix without a hyphen (e.g., `gpt-5.4` → `gpt-5.4o`), this predicate misses and per-entry settings are silently dropped. Consider using a plain `startsWith(configuredModel)` check (without the trailing hyphen) to stay consistent with fuzzy matching, or use the same fuzzy matcher here.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

@chatgpt-codex-connector
Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@RaviTharuma
Copy link
Copy Markdown
Contributor Author

Fixed and pushed in 3c0f080a.

What changed:

  • Removed unsupported OpenCode prompt payload fields from delegated/background session.prompt / session.promptAsync bodies:
    • no temperature / top_p inside body.model
    • no body.options
  • Kept only supported prompt payload fields:
    • model: { providerID, modelID }
    • top-level variant
  • Routed advanced tuning through the correct plugin path instead:
    • added per-session prompt param store in src/shared/session-prompt-params-state.ts
    • applied temperature, topP, reasoningEffort, thinking, and maxTokens via chat.params in src/plugin/chat-params.ts
    • clear stored params on session.deleted

Also fixed fallback-promotion matching:

  • exact match now wins before prefix match
  • prefix matching now uses startsWith(configuredModel) so promoted settings stay aligned with fuzzy resolution, including gpt-5.4 -> gpt-5.4o

Follow-up suite cleanup included in the same push:

  • Darwin realpath normalization now handles /private/tmp
  • config-source skill glob filtering fixed
  • processed command store pruning made amortized/in-place (fixes the 10k-entry timeout)
  • tmux env test updated to snapshot process.env once, removing the suite flake

Validation:

  • bun run typecheck
  • focused regression suite for these paths ✅
  • full suite: 4075 pass, 0 fail

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 22 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/tools/delegate-task/category-resolver.ts">

<violation number="1" location="src/tools/delegate-task/category-resolver.ts:33">
P2: Unconstrained prefix fuzzy matching can select the wrong fallback entry for models with shared prefixes (e.g., gpt-4 vs gpt-4o).</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

@chatgpt-codex-connector
Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@gitguardian
Copy link
Copy Markdown

gitguardian bot commented Mar 17, 2026

️✅ There are no secrets present in this pull request anymore.

If these secrets were true positive and are still valid, we highly recommend you to revoke them.
While these secrets were previously flagged, we no longer have a reference to the
specific commits where they were detected. Once a secret has been leaked into a git
repository, you should consider it compromised, even if it was deleted immediately.
Find here more information about risks.


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

@RaviTharuma
Copy link
Copy Markdown
Contributor Author

Follow-up: I removed the secret-like test fixture strings that triggered the GitGuardian generic-entropy detector in src/features/skill-mcp-manager/env-cleaner.test.ts and pushed the cleanup. The values are now obviously fake placeholders; no real credential from the shell auth flow was committed. GitHub may take a short time to refresh the GitGuardian status after the new push.

@chatgpt-codex-connector
Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 1 file (changes from recent commits).

Requires human review: Large PR (1.8k+ lines) modifying core model fallback and session parameter handling. Significant architectural changes to sidecar state stores warrant manual review.

@acamq
Copy link
Copy Markdown
Collaborator

acamq commented Mar 17, 2026

@cubic-dev-ai review

@cubic-dev-ai
Copy link
Copy Markdown

cubic-dev-ai bot commented Mar 17, 2026

@cubic-dev-ai review

@acamq I have started the AI code review. It will take a few minutes to complete.

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

6 issues found across 38 files

Confidence score: 3/5

  • There is moderate regression risk from concrete behavior gaps: src/config/schema/fallback-models.ts currently rejects valid Opencode/OpenAI reasoningEffort values ("none", "minimal"), which can break compatible configs at runtime.
  • src/tools/delegate-task/subagent-resolver.ts has a likely fallback resolution bug (high confidence) where built-in fallbackChain can be skipped when no user fallback models are set, so delegation may choose the wrong model path.
  • Other findings are mostly maintainability or lower-confidence concerns (duplicated parsing/type definitions in src/shared/fallback-chain-from-models.ts and src/tools/delegate-task/sync-task.ts, plus uncertain stale session params in src/tools/delegate-task/sync-prompt-sender.ts), which keeps this from being high risk but not fully low risk.
  • Pay close attention to src/config/schema/fallback-models.ts, src/tools/delegate-task/subagent-resolver.ts, and src/tools/delegate-task/sync-prompt-sender.ts - config compatibility, fallback-chain correctness, and possible stale session tuning state.
Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/config/schema/fallback-models.ts">

<violation number="1" location="src/config/schema/fallback-models.ts:6">
P1: Custom agent: **Opencode Compatibility**

Update the `reasoningEffort` enum to include `"none"` and `"minimal"`. Opencode explicitly supports these values for OpenAI models (as seen in `OPENAI_EFFORTS` inside the provider transform logic). Omitting them from the schema prevents users from configuring these legitimate fallback settings.</violation>
</file>

<file name="src/tools/delegate-task/subagent-resolver.test.ts">

<violation number="1" location="src/tools/delegate-task/subagent-resolver.test.ts:375">
P3: Test doesn’t exercise prefix-specific matching: available model exactly matches a fallback entry, so exact-match logic wins and prefix selection isn’t tested.</violation>
</file>

<file name="src/shared/fallback-chain-from-models.ts">

<violation number="1" location="src/shared/fallback-chain-from-models.ts:66">
P2: Duplicate model string parsing logic - `parseFallbackModelObjectEntry` duplicates ~10 lines of parsing code from `parseFallbackModelEntry` (trimming, splitting by `/`, provider fallback resolution, variant extraction). Refactor to call `parseFallbackModelEntry(obj.model, ...)` and merge additional properties.</violation>
</file>

<file name="src/tools/delegate-task/sync-prompt-sender.ts">

<violation number="1" location="src/tools/delegate-task/sync-prompt-sender.ts:73">
P2: Session-level prompt params are only set when tuning values exist, so later prompts without tuning for the same session leave stale temperature/topP/options in the session map and can incorrectly apply them.</violation>
</file>

<file name="src/tools/delegate-task/sync-task.ts">

<violation number="1" location="src/tools/delegate-task/sync-task.ts:20">
P2: Expanded inline `categoryModel` object type is duplicated across multiple delegate-task files, creating avoidable drift and maintainability risk; extract to a shared type/interface.</violation>
</file>

<file name="src/tools/delegate-task/subagent-resolver.ts">

<violation number="1" location="src/tools/delegate-task/subagent-resolver.ts:198">
P2: Promotion of fallback model config ignores built-in agent fallbackChain when no user fallback models are configured, because getFallbackModelConfig is called with configuredFallbackChain instead of the resolved fallbackChain.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

@chatgpt-codex-connector
Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@RaviTharuma
Copy link
Copy Markdown
Contributor Author

Follow-up on the red test check: I reviewed the failed Actions job (23203946798 / job 67433875416). The failure is not from a test assertion in the branch code; it dies during Bun binary extraction for the Windows baseline artifact (Failed to extract executable for bun-windows-x64-baseline-v1.3.10). Local verification in the isolated PR worktree is green (bun run typecheck plus the focused delegate/background/fallback suites). If someone with repo permissions can rerun the test job, this looks like a CI/download flake rather than a code regression.

@RaviTharuma
Copy link
Copy Markdown
Contributor Author

@cubic-dev-ai All active review threads on this PR are resolved and the branch head is updated (487cf198). The current red GitHub test job appears to be a CI artifact download/extraction problem rather than a code assertion failure. If you still see a code issue on the latest head, please re-review the current diff state.

@cubic-dev-ai
Copy link
Copy Markdown

cubic-dev-ai bot commented Mar 17, 2026

@cubic-dev-ai All active review threads on this PR are resolved and the branch head is updated (487cf198). The current red GitHub test job appears to be a CI artifact download/extraction problem rather than a code assertion failure. If you still see a code issue on the latest head, please re-review the current diff state.

@RaviTharuma I have started the AI code review. It will take a few minutes to complete.

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4 issues found across 38 files

Confidence score: 3/5

  • There is a concrete medium-risk behavior in src/features/opencode-skill-loader/config-source-discovery.ts: negation globs like !secret.md can be bypassed by the current OR logic between directory and file matches, so excluded files may still be discovered.
  • src/tools/delegate-task/subagent-resolver.ts has a specificity bug in fallback selection (sorting by longest provider string instead of matching-prefix length), which can route to a less-appropriate fallback in some provider-name combinations.
  • src/tools/delegate-task/sync-prompt-sender.ts may carry stale session prompt params into fallback models, but confidence is low; together with duplication in src/features/background-agent/manager.ts, this points to maintainability/regression risk rather than an immediate blocker.
  • Pay close attention to src/features/opencode-skill-loader/config-source-discovery.ts and src/tools/delegate-task/subagent-resolver.ts - exclusion correctness and fallback-target selection have the highest user-facing impact.
Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/tools/delegate-task/subagent-resolver.ts">

<violation number="1" location="src/tools/delegate-task/subagent-resolver.ts:50">
P2: Fallback matching sorts by the longest provider string in the entry rather than the length of the matching prefix, which can select a less-specific fallback when an unrelated provider name is longer.</violation>
</file>

<file name="src/features/background-agent/manager.ts">

<violation number="1" location="src/features/background-agent/manager.ts:515">
P2: Prompt-parameter normalization/mapping logic is duplicated in both launch and resume paths, increasing risk of drift when model params evolve.</violation>
</file>

<file name="src/features/opencode-skill-loader/config-source-discovery.ts">

<violation number="1" location="src/features/opencode-skill-loader/config-source-discovery.ts:43">
P2: Negation globs (e.g., `!secret.md`) are effectively bypassed because directory matches are OR’d with file matches; the negation matches non‑matching directory strings and short‑circuits to true, so explicitly excluded files are loaded.</violation>
</file>

<file name="src/tools/delegate-task/sync-prompt-sender.ts">

<violation number="1" location="src/tools/delegate-task/sync-prompt-sender.ts:64">
P2: Fallback models without explicit tuning overrides will inherit stale session prompt parameters because session params are only set when overrides exist and never cleared for a new model without overrides.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 issues found across 9 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/features/opencode-skill-loader/config-source-discovery.ts">

<violation number="1" location="src/features/opencode-skill-loader/config-source-discovery.ts:35">
P2: Extglob patterns starting with `!(...)` are misclassified as global negations, causing valid skills to be incorrectly filtered out.</violation>
</file>

<file name="src/hooks/thinking-block-validator/hook.ts">

<violation number="1" location="src/hooks/thinking-block-validator/hook.ts:119">
P1: Shared closure state (`thinkingStripped`) is used across async hook phases, creating cross-request race conditions and incorrect `chat.params` behavior under concurrency.</violation>

<violation number="2" location="src/hooks/thinking-block-validator/hook.ts:169">
P1: Removing empty assistant messages after stripping thinking blocks can create consecutive user messages, violating Anthropic API's alternating roles requirement</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

RaviTharuma added a commit to RaviTharuma/oh-my-openagent that referenced this pull request Mar 18, 2026
…blocks & drop empty messages

When the validator strips thinking blocks from history, it must also
disable the `thinking` request parameter — otherwise the API still
enforces that every assistant message starts with a thinking block,
causing 'Expected thinking but found tool_use' rejections.

Added a shared closure flag between the two hook points:
- messages.transform sets the flag when it strips
- chat.params reads the flag and sets thinking=undefined

Also filter out assistant messages left with empty parts arrays after
stripping (e.g. thinking-only responses), which would cause separate
API validation failures.

Fixes cubic-dev-ai P0 and P2 findings on PR code-yeongyu#2622.
@RaviTharuma RaviTharuma force-pushed the feat/object-style-fallback-models branch from 5137ddf to 8a9807d Compare March 18, 2026 13:20
@chatgpt-codex-connector
Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@RaviTharuma
Copy link
Copy Markdown
Contributor Author

Rebased onto latest dev — clean diff

This branch was rebased onto the latest dev to remove 459 upstream commits that were polluting the diff.

Before: 685 files changed, 491 mixed commits
After: 46 files changed, 26 focused commits

Reviewable commit groups

The 26 commits are organized by concern. You can review them in logical groups:

Group Commits Files Description
Core feature d8cfe3a..3f6b6c3 5 Object-style fallback_models config schema + model resolver
Fallback chain 3c045fc..6cc155b 6 Prefix-match lookup, chain construction, promotion logic
Delegate-task 8d480c2..051934f 13 Prompt payload alignment, subagent resolver, model selection
Model compat 75cb051..d7fb439 3 MODEL_FAMILY_REGISTRY, provider-agnostic detection, reasoning effort
Thinking blocks 83c3464..a074115 5 Strip-instead-of-inject, chat.params integration
Skill loader b1d7cfb 1 Negated glob every() fix
GPT-legacy fix e136d46 2 Remove reasoningEffort from legacy GPT models
Misc fixes 3df74f9..8a9807d 4 Review follow-ups, glob guard, test cleanup

Each commit message has a conventional-commits scope that maps to these groups.

@RaviTharuma RaviTharuma force-pushed the feat/object-style-fallback-models branch from 8a9807d to 36d7949 Compare March 18, 2026 13:25
@chatgpt-codex-connector
Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

RaviTharuma added a commit to RaviTharuma/oh-my-openagent that referenced this pull request Mar 18, 2026
@RaviTharuma
Copy link
Copy Markdown
Contributor Author

@cubic-dev-ai please review

@cubic-dev-ai
Copy link
Copy Markdown

cubic-dev-ai bot commented Mar 18, 2026

@cubic-dev-ai please review

@RaviTharuma I have started the AI code review. It will take a few minutes to complete.

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 34 files

Confidence score: 4/5

  • This PR looks safe to merge overall; the reported issue is moderate (5/10) and describes a maintainability/regression risk rather than an immediate functional break.
  • In src/features/background-agent/spawner.ts, duplicated prompt-parameter mapping across startTask and resumeTask could drift over time, causing inconsistent model behavior when settings are updated.
  • Pay close attention to src/features/background-agent/spawner.ts - duplicated mapping logic should be centralized to reduce future desync risk.
Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/features/background-agent/spawner.ts">

<violation number="1" location="src/features/background-agent/spawner.ts:139">
P2: Duplicated model prompt-parameter mapping in `startTask` and `resumeTask` creates a desync risk when model settings evolve.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

Copy link
Copy Markdown
Contributor Author

@RaviTharuma RaviTharuma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re: cubic-dev-ai review — Violation 1 (P2): Duplicated model prompt-parameter mapping

This is already addressed in the current code. The applyModelPromptParams helper (lines 21–42 of spawner.ts) was extracted exactly to centralize the mapping logic. Both startTask (line 164) and resumeTask (line 256) call the same shared function — no duplication.

The remaining parallel structure (launchModel/resumeModel and launchVariant/resumeVariant) is inherent to the two call sites having different input types (LaunchInput vs BackgroundTask). These are simple field extractions, not duplicated business logic, and they can't drift independently from the shared applyModelPromptParams.

No changes needed.

@RaviTharuma
Copy link
Copy Markdown
Contributor Author

@acamq please check it again :)

@code-yeongyu code-yeongyu added the triage:feature-request Feature or enhancement request label Mar 24, 2026
@RaviTharuma RaviTharuma force-pushed the feat/object-style-fallback-models branch from 36d7949 to 1de5b66 Compare March 25, 2026 08:53
@RaviTharuma
Copy link
Copy Markdown
Contributor Author

@code-yeongyu I restacked this branch onto current dev as 1de5b66a, so the head is no longer on the old March 18 base. I used your already-restacked resolver branch on #2643 as the source of truth for the upper stack and then rebuilt #2674 on top of it. Verification was run on the full restacked stack head in #2674: bun test src/shared/model-settings-compatibility.test.ts src/plugin/chat-params.test.ts src/features/background-agent/spawner.test.ts src/features/background-agent/manager.test.ts src/tools/delegate-task/sync-prompt-sender.test.ts -> 186 pass, 0 fail; bunx tsc --noEmit -> clean.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1de5b66a8a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector
Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

…tings

Add support for object-style entries in fallback_models arrays, enabling
per-model configuration of variant, reasoningEffort, temperature, top_p,
maxTokens, and thinking settings.

- Zod schema for FallbackModelObject with full validation
- normalizeFallbackModels() and flattenToFallbackModelStrings() utilities
- Provider-agnostic model resolution pipeline with fallback chain
- Session prompt params state management
- Fallback chain construction with prefix-match lookup
- Integration across delegate-task, background-agent, and plugin layers
Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 8 files (changes from recent commits).

Requires human review: Large change (2800+ lines) affecting core model resolution, prompt payloads, and state management. High complexity makes 100% certainty against regressions difficult without manual verification.

@RaviTharuma RaviTharuma force-pushed the feat/object-style-fallback-models branch from a841eac to fb08553 Compare March 25, 2026 10:15
@chatgpt-codex-connector
Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@RaviTharuma
Copy link
Copy Markdown
Contributor Author

@code-yeongyu restacked this branch onto the current dev head (7761e48d) and force-pushed a new head: fb085538.

The rebase exposed one real failure on this branch: src/features/background-agent/spawner.test.ts had lost the createTask import, which broke the keeps agent when explicit model is configured case. That is fixed on this head.

Local verification on fb085538:

  • bunx tsc --noEmit
  • CI-equivalent test workload rerun locally: 2580 pass, 0 fail
  • bun run build

Fresh CI run for this push: 23535863824 (in progress).

@RaviTharuma
Copy link
Copy Markdown
Contributor Author

@code-yeongyu final status on this stack after the latest restack:

  • base is current dev (7761e48d)
  • head is fb085538
  • local verification passed (bunx tsc --noEmit, bun run build, full local CI-equivalent test run)
  • fresh GitHub CI is now green
  • GitHub merge state is now clean

From my side this PR is ready for your normal review/merge flow as stack item 1 of 3.

@code-yeongyu code-yeongyu merged commit d6d4cec into code-yeongyu:dev Mar 25, 2026
7 checks passed
@RaviTharuma RaviTharuma deleted the feat/object-style-fallback-models branch March 28, 2026 08:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

triage:feature-request Feature or enhancement request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

runtime-fallback: fallback models lose variant and reasoningEffort settings

3 participants