refactor: Defer loading provider catalog at startup - #52816

Hey! Your PR title Defer loading provider catalog at startup doesn't follow conventional commit format.

Please update it to start with one of:

• feat: or feat(scope): new feature
• fix: or fix(scope): bug fix
• docs: or docs(scope): documentation changes
• chore: or chore(scope): maintenance tasks
• refactor: or refactor(scope): code refactoring
• test: or test(scope): adding or updating tests

Where scope is the package name (e.g., app, desktop, opencode).

See CONTRIBUTING.md for details.

The following comment was made by an LLM, it may be inaccurate:

Thanks for updating your PR! It now meets our contributing guidelines. 👍

I measured this while looking at startup, and the win is real but a lot smaller than the 850 ms in the description — worth having the number before it is prioritised.

Method: build the CLI twice, once with the real catalog and once with an empty one, and time the compiled binary.

$ MODELS_DEV_API_JSON=/tmp/empty.json bun packages/opencode/script/build.ts --single --skip-embed-web-ui
$ ./packages/opencode/dist/opencode-darwin-arm64/bin/opencode --version   # N runs, wall clock
$ ./packages/opencode/dist/opencode-darwin-arm64/bin/opencode serve --port 4200   # until "listening"

Apple M4 Pro, macOS 27.0.1, bun 1.4.2, catalog api.json 5.3 MB / 226 providers:

So the whole catalog costs ~27 ms at CLI load and ~68 ms at server boot on this machine. For reference, JSON.parse of the 5.3 MB api.json alone is 13 ms (n=3), so the 850 ms is likely the decode/instantiation rather than the parse — which makes deferring it the right shape of fix, just a smaller one.

For scale, the CLI command graph currently costs more: a build of src/index.ts with the .command(...) registrations removed comes in at 113 ms for --version versus 301 ms, so ~188 ms of that 301 ms is modules the path never uses.

I opened #53593 for the same thing before I found this one, sorry for the overlap.

One case this misses: the model picker also uses createDialogProviderOptions(), and when no provider is connected it shows "Popular providers" from provider_next. With this PR that section stays empty until /connect has been opened once. In mine I call the loader from createDialogProviderOptions on mount, so both dialogs get it.

On the size question above: the TUI fetches /provider on every bootstrap, and that includes every time it reloads for another directory, not only at startup. Here it's 6.7 MB and 185–420 ms on a warm server. Over 24 session switches across directories, the blocking part of that reload went from a median 495 ms to 87 ms without it.

Happy to close mine if you add the model picker case, or the maintainers can take whichever they prefer.

I measured this while looking at startup, and the win is real but a lot smaller than the 850 ms in the description — worth having the number before it is prioritised.

Method: build the CLI twice, once with the real catalog and once with an empty one, and time the compiled binary.

$ MODELS_DEV_API_JSON=/tmp/empty.json bun packages/opencode/script/build.ts --single --skip-embed-web-ui
$ ./packages/opencode/dist/opencode-darwin-arm64/bin/opencode --version   # N runs, wall clock
$ ./packages/opencode/dist/opencode-darwin-arm64/bin/opencode serve --port 4200   # until "listening"

Apple M4 Pro, macOS 27.0.1, bun 1.4.2, catalog api.json 5.3 MB / 226 providers:
build --version (median, n=10) serve → listening (median, n=6)
real catalog 301 ms 497 ms
empty catalog 274 ms 429 ms

So the whole catalog costs ~27 ms at CLI load and ~68 ms at server boot on this machine. For reference, JSON.parse of the 5.3 MB api.json alone is 13 ms (n=3), so the 850 ms is likely the decode/instantiation rather than the parse — which makes deferring it the right shape of fix, just a smaller one.

For scale, the CLI command graph currently costs more: a build of src/index.ts with the .command(...) registrations removed comes in at 113 ms for --version versus 301 ms, so ~188 ms of that 301 ms is modules the path never uses.

I was running this on a xen VM with limited memory - so my parse times are probably significantly longer.

I opened #53593 for the same thing before I found this one, sorry for the overlap.

One case this misses: the model picker also uses createDialogProviderOptions(), and when no provider is connected it shows "Popular providers" from provider_next. With this PR that section stays empty until /connect has been opened once. In mine I call the loader from createDialogProviderOptions on mount, so both dialogs get it.

On the size question above: the TUI fetches /provider on every bootstrap, and that includes every time it reloads for another directory, not only at startup. Here it's 6.7 MB and 185–420 ms on a warm server. Over 24 session switches across directories, the blocking part of that reload went from a median 495 ms to 87 ms without it.

Happy to close mine if you add the model picker case, or the maintainers can take whichever they prefer.

If yours solves the issue, I'm happy to close this in favor of yours. I was just tired of the slow startup times I was seeing.

This branch has been upgraded to V2. We're no longer taking PRs for V1. If you think this is still relevant, please port it over and open it for V2.

— from 𝕺𝖕𝖊𝖓𝕮𝖔𝖉𝖊