-
Notifications
You must be signed in to change notification settings - Fork 16.5k
Closed
Labels
good first issuekind:bugThis is a clearly a bugThis is a clearly a bugprovider:googleGoogle (including GCP) related issuesGoogle (including GCP) related issues
Description
Apache Airflow version
main (development)
What happened
The Bigquery hook get_records always return a list of string irrespective of the Bigquery table column type. So if even my table column has value 0 the task succeeds since boo("0") return True
https://github.com/apache/airflow/blob/main/airflow/providers/google/cloud/hooks/bigquery.py#L3062
What you think should happen instead
Bigquery hook get_records should return the value having the correct col type
How to reproduce
create an Airflow google cloud connection google_cloud_default and try the below DAG.
Make sure to update the DATASET and Table name.
Your table first row should contain a 0 value in this case expected behaviour is DAG should fail but it will pass
from datetime import datetime, timedelta
from airflow import DAG
from airflow.providers.google.cloud.operators.bigquery import BigQueryCheckOperator
default_args = {
"execution_timeout": timedelta(minutes=30),
}
with DAG(
dag_id="bq_check_op",
start_date=datetime(2022, 8, 22),
schedule_interval=None,
catchup=False,
default_args=default_args,
tags=["example", "async", "bigquery"],
) as dag:
check_count = BigQueryCheckOperator(
task_id="check_count",
sql=f"SELECT * FROM DATASET.TABLE",
use_legacy_sql=False,
deferrable=True
)
```an
### Operating System
Mac
### Versions of Apache Airflow Providers
_No response_
### Deployment
Docker-Compose
### Deployment details
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
good first issuekind:bugThis is a clearly a bugThis is a clearly a bugprovider:googleGoogle (including GCP) related issuesGoogle (including GCP) related issues