-
Notifications
You must be signed in to change notification settings - Fork 3.2k
feat: multi model and multi provider config and auto switching (lead/worker consolidation/simplification) #4036
Description
WIP PR: #4035
starting with just the mechanism, as there are other changes to unify the config I don't wnat to get in the way of
Need to be able to configure multiple models and providers for different purposes, and untangle lead/worker config and code.
Discussed in #3980
Originally posted by michaelneale August 9, 2025
I started looking at how we can consolidate things around lead/worker and provider config (vs env vars), lead worker is currently special cased (and wont' support streaming for example), and uses environment variables, a whole variety of confusing things.
What we could do is have a more general config something like:
models:
- provider: grok
model: grok4
purpose: oracle
- provider: openai
model: gpt-5
purpose: second-opinion
- provider: anthropic
model: claude-sonnet-4-20250514
purpose: worker
- provider: anthropic
model: claude-opus-4-20250514
purpose: leadwhere there are multiple providers configured for different purposes. Lead/worker is just one case (it could have just one by default, similar to before).
oracle is for one of consultation, second-opinion for when assistance is needed and so on (can add extra metadata to help the routing/gating decisions, but don't want to get bogged in that detail).
This would also go along with simplifying the lead/worker code to be less of a special case, and more about tracking what provider is in use and switching, and loading config from one place ideally (and not only rely on environment variables).
While making that change, this can also use a simpler config format (of course for compat/command line etc - can still have env vars set when that isn't used)