Skip to content

BigQueryCheckOperator fail in deferrable mode even if col val 0 #27237

@pankajastro

Description

@pankajastro

Apache Airflow version

main (development)

What happened

The Bigquery hook get_records always return a list of string irrespective of the Bigquery table column type. So if even my table column has value 0 the task succeeds since boo("0") return True
https://github.com/apache/airflow/blob/main/airflow/providers/google/cloud/hooks/bigquery.py#L3062

What you think should happen instead

Bigquery hook get_records should return the value having the correct col type

How to reproduce

create an Airflow google cloud connection google_cloud_default and try the below DAG.
Make sure to update the DATASET and Table name.
Your table first row should contain a 0 value in this case expected behaviour is DAG should fail but it will pass

from datetime import datetime, timedelta

from airflow import DAG
from airflow.providers.google.cloud.operators.bigquery import BigQueryCheckOperator

default_args = {
    "execution_timeout": timedelta(minutes=30),
}


with DAG(
    dag_id="bq_check_op",
    start_date=datetime(2022, 8, 22),
    schedule_interval=None,
    catchup=False,
    default_args=default_args,
    tags=["example", "async", "bigquery"],
) as dag:
    check_count = BigQueryCheckOperator(
        task_id="check_count",
        sql=f"SELECT * FROM DATASET.TABLE",
        use_legacy_sql=False,
        deferrable=True
    )

```an

### Operating System

Mac

### Versions of Apache Airflow Providers

_No response_

### Deployment

Docker-Compose

### Deployment details

_No response_

### Anything else

_No response_

### Are you willing to submit PR?

- [X] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions