future annotations with Python 3.8 cause TypeError

Initial Checks

I confirm that I'm using Pydantic V2

Description

Using __future__ import annotations with Python3.8 and Pydantic 2.4.2 cause:

TypeError: unsupported operand type(s) for |: 'type' and 'type'

Example Code

from __future__ import annotations
from pydantic import BaseModel

class Bios(BaseModel):
    value: int | str

def p(d: int | str | Bios) -> None:
    print(d)

if __name__ == '__main__':
    p('a')
    p(1)
    p(Bios(value=2))
    p(Bios(value='s'))

Python, Pydantic & OS Version

pydantic version: 2.4.2
pydantic-core version: 2.10.1
pydantic-core build: profile=release pgo=false
install path: C:\Users\mplic\Projects\venvs\dcspy_38\Lib\site-packages\pydantic
python version: 3.8.10 (tags/v3.8.10:3d8993a, May  3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)]
platform: Windows-10-10.0.19045-SP0
related packages: mypy-1.6.1 typing_extensions-4.7.1

@emcek

Hi @emcek,

Thanks for reporting this! I was able to replicate this issue. Indeed seems like a bug 🐛. We will look into a fix!

I can confirm I have the same bug when using Python 3.9.

Python, Pydantic & OS Version

pydantic version: 2.4.2
pydantic-core version: 2.10.1
pydantic-core build: profile=release pgo=false
install path: /Users/benroland/python_projects/mltoolbox/.venv/lib/python3.9/site-packages/pydantic
python version: 3.9.18 (main, Oct 20 2023, 11:16:30)  [Clang 15.0.0 (clang-1500.0.40.1)]
platform: macOS-13.6-arm64-arm-64bit
related packages: mypy-1.6.1 typing_extensions-4.8.0

I feel like I'm missing something...is this just expected to work? Even with __future__ import annotations, pydantic needs to actually evaluate int | str at runtime, and it's completely normal that before Python 3.10 this raises an error. This is a limitation of older versions of Python for any program that needs type hints at runtime, not just pydantic. Same for using list[str] instead of typing.List[str].

Maybe this could be made to work, but it seems more like a feature request than a bug.

The solution I'm imagining is modifying the AST to replace | with Union before evaluating. I think it'll work, I'm just not sure if it's worth the complexity to make it slightly more convenient to use older Python versions.

The solution I'm imagining is modifying the AST to replace | with Union before evaluating. I think it'll work, I'm just not sure if it's worth the complexity to make it slightly more convenient to use older Python versions.

Yes, Union will work. This is a way I discover this issue try migrate from Union to | for Python 3.8 using future import.

@alexmojaki

@alexmojaki @emcek,

I'm inclined to think this is a bug with Pydantic due to PEP 563 and the pydantic docs on postponed annotations.

OK, then I'll take this one.

@alexmojaki

@alexmojaki Awesome! I'll take a look at your PRs when I have a chance today / tmrw. Thanks for all of the contributions 😄.

@alexmojaki

@alexmojaki ping me when PR is ready I would love to check solution...

I feel like I'm missing something...is this just expected to work? Even with __future__ import annotations, pydantic needs to actually evaluate int | str at runtime, and it's completely normal that before Python 3.10 this raises an error. This is a limitation of older versions of Python for any program that needs type hints at runtime, not just pydantic. Same for using list[str] instead of typing.List[str].

Maybe this could be made to work, but it seems more like a feature request than a bug.

I agree. This is why I made a POC+ in https://github.com/PrettyWood/future-typing but it still needs some work and IMO is not worth the time. I don't see how it should work without that

What I'm thinking is transforming the AST and then compiling and evaluating that directly, which should be easier than manipulating tokens.

Ah yes ! I'm still stuck in v1 where we depended on cython that made evaluation impossible

PR ready: #7958

patch pydantic runtime type support

Remove Python 3.8 ci, see pydantic/pydantic/issues/7873

devel/py-eval-type-backport: Add py-eval-type-backport 0.2.2

Navigation Menu

Repository navigation

future annotations with Python 3.8 cause TypeError #7873

Initial Checks

Description

Example Code

Python, Pydantic & OS Version

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

future annotations with Python 3.8 cause TypeError #7873

Description

Initial Checks

Description

Example Code

Python, Pydantic & OS Version

Activity

sydney-runkle commented on Oct 19, 2023

Parici75 commented on Oct 20, 2023

alexmojaki commented on Oct 20, 2023

alexmojaki commented on Oct 21, 2023

emcek commented on Oct 21, 2023

sydney-runkle commented on Oct 23, 2023

alexmojaki commented on Oct 23, 2023

sydney-runkle commented on Oct 23, 2023

emcek commented on Oct 23, 2023

PrettyWood commented on Oct 24, 2023

alexmojaki commented on Oct 24, 2023

PrettyWood commented on Oct 24, 2023

alexmojaki commented on Nov 4, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions