Skip to content

Allow use of beta Anthropic models #4357

@Dan-Wuensch

Description

@Dan-Wuensch

Please explain the motivation behind the feature request.

Although it's possible to use Anthropic beta models in AWS Bedrock that provide much higher token limits or other capabilities that aren't available in base models, this option isn't available in Goose yet.

Describe the solution you'd like

Allow setting an optional BEDROCK_ANTHROPIC_BETA environment variable for the Bedrock provider which would, if provided, result in the following being set in request_body when making an API call to Bedrock

request_body = {
    "additionalModelRequestFields": {
        "anthropic_beta": ["context-1m-2025-08-07"]
    }
}

Reference: https://aws.plainenglish.io/use-1m-tokens-with-claude-sonnet-in-aws-bedrock-73a46af45d6c

If Goose keeps internal counters of tokens available that affect functionality, these may need to also be increased in the advent that a beta model is requested with higher tokens. Available tokens is a separate albieit related parameter - we may need to either set an existing setting for that or introduce a new one.

Describe alternatives you've considered
Been working on #4173 as a way to avoid failed sessions due to AWS's aggressive Bedrock throttling. The combination of this and access to higher token models will really help avoid these kinds of failures.

However even without throttling being an issue, access to more tokens can increase capabilities in useful ways!

Normal:
Cross-region model inference tokens per minute for Anthropic Claude Sonnet 4 V1 - 200,000
Model invocation max tokens per day for Anthropic Claude Sonnet 4 V1 (doubled for cross-region calls) - 144,000,000

1M:
Cross-region model inference tokens per minute for Anthropic Claude Sonnet 4 V1 1M Context Length - 1,000,000
Model invocation max tokens per day for Anthropic Claude Sonnet 4 V1 1M Context Length (doubled for cross-region calls) - 720,000,000

Additional context
Add any other context or screenshots about the feature request here.

  • I have verified this does not duplicate an existing feature request

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestp3Priority 3 - Low

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions