-
Notifications
You must be signed in to change notification settings - Fork 3.3k
Allow use of beta Anthropic models #4357
Description
Please explain the motivation behind the feature request.
Although it's possible to use Anthropic beta models in AWS Bedrock that provide much higher token limits or other capabilities that aren't available in base models, this option isn't available in Goose yet.
Describe the solution you'd like
Allow setting an optional BEDROCK_ANTHROPIC_BETA environment variable for the Bedrock provider which would, if provided, result in the following being set in request_body when making an API call to Bedrock
request_body = {
"additionalModelRequestFields": {
"anthropic_beta": ["context-1m-2025-08-07"]
}
}
If Goose keeps internal counters of tokens available that affect functionality, these may need to also be increased in the advent that a beta model is requested with higher tokens. Available tokens is a separate albieit related parameter - we may need to either set an existing setting for that or introduce a new one.
Describe alternatives you've considered
Been working on #4173 as a way to avoid failed sessions due to AWS's aggressive Bedrock throttling. The combination of this and access to higher token models will really help avoid these kinds of failures.
However even without throttling being an issue, access to more tokens can increase capabilities in useful ways!
Normal:
Cross-region model inference tokens per minute for Anthropic Claude Sonnet 4 V1 - 200,000
Model invocation max tokens per day for Anthropic Claude Sonnet 4 V1 (doubled for cross-region calls) - 144,000,000
1M:
Cross-region model inference tokens per minute for Anthropic Claude Sonnet 4 V1 1M Context Length - 1,000,000
Model invocation max tokens per day for Anthropic Claude Sonnet 4 V1 1M Context Length (doubled for cross-region calls) - 720,000,000
Additional context
Add any other context or screenshots about the feature request here.
- I have verified this does not duplicate an existing feature request