Skip to content

fix(server): serve last settled models from /api/model while plugins reload - #57011

Draft
kitlangton wants to merge 1 commit into
devfrom
model-reload-fallback
Draft

kitlangton wants to merge 1 commit into
devfrom
model-reload-fallback

Conversation

@kitlangton

Copy link
Copy Markdown
Contributor

Issue for this PR

N/A

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Serves the last settled model catalog and default model from GET /api/model and GET /api/model/default while plugins/config are reloading instead of blocking on awaitPlugins and failing with 503 ServiceUnavailableError after 5 s.

When global config (opencode.json) is edited on a server with many loaded Locations (e.g. 133 booted Locations in our Studio server trace), PluginSupervisor holds Plugin.awaitActivation across all loaded Locations while debouncing and re-running Plugin.activate inside State.batch. During that reload window:

  1. Plugin.awaitActivation can stay pending for >5 s across the loaded Location set (5.1–11.6 s in opencode.log), causing awaitPlugins in ModelHandler to fail GET /api/model with HTTP 503 (Location plugins did not finish activating within 5 seconds).
  2. Inside Plugin.activate's State.batch, closing a changed plugin's scope removes its provider.transform / model.transform registrations and invalidates Provider.Service / Model.Service before the replacement finishes loading, so rebuilding on the HTTP fiber mid-batch would observe an incomplete catalog.

Changes in packages/server/src/handlers/model.ts:

  • Cache the last settled { available, default } snapshot per Model.Interface in a WeakMap whenever Plugin.awaitActivation is open (Duration.zero check).
  • When Plugin.awaitActivation is pending on a Location that has already settled a snapshot, immediately return that last settled snapshot (model.updated notifies clients once the reload batch finishes).
  • On a cold Location that has not yet settled its first snapshot, continue waiting on awaitPlugins, and if the 5 s wait times out, fall back to the currently available models rather than returning 503.

How did you verify your code works?

cd packages/server && bun test test/model.test.ts && bun run typecheck
  • Preserves the existing cold-start test (waits for plugin initialization before listing models).
  • Adds serves the last settled model list and default model while plugins reload, which holds a plugin reload mid-batch after v1 scope teardown, verifies GET /api/model and GET /api/model/default return 200 with chat-v1 immediately, then releases the reload gate and verifies both endpoints return chat-v2.

Screenshots / recordings

N/A

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant