fix: avoid stale client references in model and embedding classes by Deeven-Seru · Pull Request #4276 · pydantic/pydantic-ai

Deeven-Seru · 2026-02-10T05:47:19Z

Closes Model classes hold stale client references when provider is re-initialized #4336

Pre-Review Checklist

Any AI generated code has been reviewed line-by-line by the human PR author, who stands by it.
No breaking changes in accordance with the version policy.
Linting and type checking pass per make format and make typecheck.
PR title is fit for the release changelog.

Pre-Merge Checklist

New tests for any fix or new behavior, maintaining 100% coverage.
Updated documentation for new features and behaviors, including docstrings for API docs.

Summary

Model and embedding classes previously cached a reference to the provider's client via self.client = provider.client / self._client = provider.client in __init__. This is a problem in environments where a provider may be re-initialized or swapped after construction (e.g. durable execution frameworks like Temporal, or tests with dependency injection): the model/embedding keeps using the stale client.

Changes

Replaced the cached client dataclass field + init assignment with a @property that delegates to self._provider.client in every affected class:
- Models (11): OpenAIChatModel, OpenAIResponsesModel, AnthropicModel, GroqModel, MistralModel, GoogleModel, GeminiModel, CohereModel, HuggingFaceModel, BedrockConverseModel, XaiModel.
- Embeddings (4): OpenAIEmbeddingModel, GoogleEmbeddingModel, CohereEmbeddingModel, BedrockEmbeddingModel.
GeminiModel.base_url now derives from self.client.base_url instead of a cached self._url, matching the other models.
Added assert model.client is provider.client identity checks to each model's existing test_init (or a new minimal test where none existed), replacing an earlier draft that used a _SwappableProvider subclass + private _client mutation.

gemini-code-assist · 2026-02-10T05:47:33Z

Summary of Changes

Hello @DEVELOPER-DEEVEN, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the robustness of the OpenAIChatModel by modifying how it interacts with its underlying OpenAI client. By transitioning from a static client assignment to a dynamic property-based retrieval, the model is guaranteed to always utilize the most up-to-date client instance. This is particularly beneficial in complex or long-running environments where provider states might change, thereby eliminating potential issues arising from outdated client connections.

Highlights

Client Access Refactor: The OpenAIChatModel now uses a property for accessing the OpenAI client, replacing the direct assignment in the constructor.
Stale Reference Prevention: This change ensures that the model always retrieves the current client from its provider, preventing stale client references, especially in durable execution environments.

Changelog

pydantic_ai_slim/pydantic_ai/models/openai.py
- Removed the direct assignment of self.client in the __init__ method.
- Introduced a new @property decorator for client that dynamically fetches the client from self._provider.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

github-actions · 2026-02-10T05:51:02Z

This PR is missing the required PR template. Please update the PR description to include:

A linked issue (using Closes #<issue>). Per the project guidelines, even simple changes should have an associated issue first to allow for discussion about whether the change is the right approach. The code change mentions durable execution environments like Temporal, but I couldn't find any existing issue about stale client references.
The pre-review checklist from the PR template, including the checkbox for AI-generated code.

Additionally:

Tests are required: The PR template states "New tests for any fix or new behavior, maintaining 100% coverage." If this fixes a real problem with stale client references, a test demonstrating the issue would help validate the fix.
Separate concerns: This PR contains two unrelated changes (the client property fix and the stream consumption docs). These should be separate PRs to keep each focused and easier to review.

DouweM · 2026-02-11T00:19:01Z

@DEVELOPER-DEEVEN Sorry for the many AI review comments, we're trying different tools and enabled more than we'd meant to at the same time... There are some good points among them though :) Let me know if you're not sure how to move forward.

Deeven-Seru · 2026-02-11T04:02:56Z

no problem @DouweM those comments are quite helpful, ill let you know if I hit any blocker

DouweM · 2026-02-16T19:33:22Z

@DEVELOPER-DEEVEN I agree with the bot's point at #4276 (comment): unless this is really specific to OpenAIChatModel, I'd rather address this everywhere at the same time.

devin-ai-integration

Devin Review found 0 new potential issues.

View 7 additional findings in Devin Review.

devin-ai-integration

Devin Review found 0 new potential issues.

View 9 additional findings in Devin Review.

devin-ai-integration

Devin Review found 0 new potential issues.

View 9 additional findings in Devin Review.

Deeven-Seru · 2026-02-17T09:10:44Z

Addressed the CI blockers in one focused commit.

fixed pyright/lint issues in client property tests
added XaiModel client passthrough coverage
closed remaining uncovered lines in the new test file

Branch updated and ready for CI/review.

Deeven-Seru · 2026-02-20T14:27:27Z

@DouweM PTAL

Deeven-Seru · 2026-03-08T14:51:52Z

Thanks for the review @DouweM — I’ve made commit and aligned with AGENTS.md: no private attribute access, tests live in existing model files, and the GoogleProvider fixture stays real (the swappable subclass is local to the one test). The PR is now focused on the client‑property change only. CI rerunning after a flaky CDN fetch; once green, PTAL.

DouweM · 2026-04-22T21:13:12Z

Sorry for the radio silence here, it's in a good state so I'll get it over the line from here!

Extend the client-property fix to `BedrockEmbeddingModel`, `OpenAIEmbeddingModel`, `GoogleEmbeddingModel`, and `CohereEmbeddingModel`, which had the same stale-reference pattern. Drop the per-test `_SwappableProvider` subclass + private `_client` mutation in favor of a single `model.client is provider.client` identity check, folded into each file's existing `test_init` where one exists.

Copilot

Pull request overview

This PR prevents model instances from holding stale SDK client references by removing client caching at construction time and instead delegating client access to the current provider client.

Changes:

Replaced cached client attributes with @property accessors delegating to self._provider.client across multiple model classes.
Updated Gemini/OpenAI/etc. model implementations to rely on the delegated client (and derived properties like base_url) rather than cached values.
Updated/added tests to assert that models expose the provider’s client.

Reviewed changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
`pydantic_ai_slim/pydantic_ai/models/openai.py`	Removes cached client fields and adds delegating `client` properties for OpenAI chat and responses models.
`pydantic_ai_slim/pydantic_ai/models/anthropic.py`	Replaces cached client with a delegating `client` property.
`pydantic_ai_slim/pydantic_ai/models/groq.py`	Replaces cached client with a delegating `client` property.
`pydantic_ai_slim/pydantic_ai/models/mistral.py`	Replaces cached client with a delegating `client` property.
`pydantic_ai_slim/pydantic_ai/models/google.py`	Replaces cached client with a delegating `client` property.
`pydantic_ai_slim/pydantic_ai/models/gemini.py`	Replaces cached client/base URL with delegating properties.
`pydantic_ai_slim/pydantic_ai/models/cohere.py`	Replaces cached client with a delegating `client` property.
`pydantic_ai_slim/pydantic_ai/models/huggingface.py`	Replaces cached client with a delegating `client` property.
`pydantic_ai_slim/pydantic_ai/models/bedrock.py`	Replaces cached Bedrock runtime client with a delegating `client` property (cast from provider).
`pydantic_ai_slim/pydantic_ai/models/xai.py`	Removes cached client assignment and adds a delegating `client` property.
`pydantic_ai_slim/pydantic_ai/embeddings/openai.py`	Switches from cached `_client` to a delegating `_client` property.
`pydantic_ai_slim/pydantic_ai/embeddings/google.py`	Switches from cached `_client` to a delegating `_client` property.
`pydantic_ai_slim/pydantic_ai/embeddings/cohere.py`	Switches from cached `_client`/`_v1_client` to delegating properties.
`pydantic_ai_slim/pydantic_ai/embeddings/bedrock.py`	Replaces cached Bedrock runtime client with a delegating `client` property.
`tests/models/test_openai.py`	Updates init test to keep provider reference and asserts `client` delegates.
`tests/models/test_openai_responses.py`	Updates test to pass explicit provider and asserts delegation.
`tests/models/test_anthropic.py`	Updates init test to assert delegation.
`tests/models/test_groq.py`	Updates init test to assert delegation.
`tests/models/test_mistral.py`	Updates init test to assert delegation.
`tests/models/test_google.py`	Adds a test asserting delegation.
`tests/models/test_gemini.py`	Adds a test asserting delegation and base_url consistency.
`tests/models/test_cohere.py`	Updates init test to assert delegation.
`tests/models/test_huggingface.py`	Adds a test asserting delegation.
`tests/models/test_bedrock.py`	Adds a test asserting delegation.
`tests/models/test_xai.py`	Updates init test to assert delegation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-23T23:15:26Z

+    provider = OpenAIProvider(api_key='foobar')
+    m = OpenAIChatModel('gpt-4o', provider=provider)
    assert m.base_url == 'https://api.openai.com/v1/'
+    assert m.client is provider.client
    assert m.client.api_key == 'foobar'


The new assertion only checks that model.client equals provider.client immediately after construction, which would also pass with the old implementation that cached provider.client in __init__. To validate the stale-reference fix, extend this test (or add a dedicated one) to mutate/refresh the provider's client after model creation (e.g., swap provider._client/recreate the provider client, or use a small stub Provider whose client property can be reassigned) and assert m.client reflects the updated client.

Copilot · 2026-04-23T23:15:27Z

        if isinstance(provider, str):
            provider = infer_provider(provider)
        self._provider = provider
-        self._client = provider.client

        super().__init__(settings=settings)

+    @property
+    def _client(self) -> AsyncOpenAI:
+        return self._provider.client
+


PR description focuses on updating model classes, but this change also alters embedding models by switching from a cached _client attribute to a delegating property. Please update the PR description (and/or add a brief note in release notes if applicable) to reflect the embeddings changes, or confirm they’re intentionally included in this fix.

Deeven-Seru · 2026-04-24T03:04:15Z

Thanks for the review and for getting this over the line, @DouweM.
Appreciate the guidance throughout the process — learned a lot while iterating on this fix.
Happy to follow up on any future improvements if needed.

github-actions Bot added the size: S Small PR (≤100 weighted lines) label Feb 10, 2026

github-actions Bot added the bug Report that something isn't working, or PR implementing a fix label Feb 10, 2026