-
Notifications
You must be signed in to change notification settings - Fork 16.6k
Description
Apache Airflow version
3.0.2
If "Other Airflow 2 version" selected, which one?
No response
What happened?
When numpy is upgraded to 2.2.6. The numpy.bool_ objects cannot be serialized/deserialized. It is because the qualified name is resolved to numpy.bool instead of numpy.bool_ while the serializers scan for numpy.bool_. As shown below:
| "numpy.bool_", |
Therefore, this object is not matched to any of the serializer and return TypeError: cannot serialize object of type <class 'numpy.bool'>
When numpy version is 1.26.4, the qualified name is resolved to numpy.bool_

When numpy version is 2.2.6, the qualified name is resolved to numpy.bool

What you think should happen instead?
numpy.bool_ objects should be serialized/deserialized when it is passed through XComs.
How to reproduce
The DAG code to reproduce the error.
from airflow.decorators import dag, task
from pendulum import datetime
@dag(
start_date=datetime(2025, 5, 23),
schedule=None,
catchup=False,
tags=["serialization", "airflow"],
)
def pydantic_serde():
# 5. NumPy scalar types - misc
@task
def get_numpy_misc():
import numpy as np
print(np.__version__)
print(type(np.bool_(False)))
print(np.bool_.__name__)
return {
"bool_": np.bool_(False),
"float16": np.float16(0.125),
"float64": np.float64(3.14159),
"complex64": np.complex64(1 + 2j),
"complex128": np.complex128(3 + 4j),
}
@task
def print_numpy_misc(obj):
print("NumPy Misc Types:")
for k, v in obj.items():
print(f"{k}: {v} ({type(v)})")
# DAG chaining
print_numpy_misc(get_numpy_misc())
pydantic_serde()Operating System
Ubuntu 22.04.3 LTS
Versions of Apache Airflow Providers
No response
Deployment
Docker-Compose
Deployment details
I run Airflow in Docker by following the guide. We need to update the docker-compose.yaml to build from the following Docker image. We can create a requirements.txt to install numpy 2.2.6
https://airflow.apache.org/docs/apache-airflow/stable/howto/docker-compose/index.html
FROM apache/airflow:3.0.2
COPY requirements.txt requirements.txt
RUN pip install --no-cache-dir -r requirements.txt
USER ${AIRFLOW_UID}requirements.txt
numpy==2.2.6
Anything else?
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct