Apache Airflow version
Other Airflow 2 version (please specify below)
What happened
I believe that the @task.short_circuit operator is missing the code to handle the usage of templates_dict, as described in the documentation. The code snippet below demonstrates the issue. In it, I create two tasks. The first is a basic python task, and in the log, from the print statement, I see the contents of the .sql file. (So the templating worked correctly.) However, in the log for the second task, I see only the string itself, with no jinja templating performed. I think this contradicts the documentation.
from airflow.decorators import dag, task
import datetime as dt
@dag(
dag_id='test_dag',
schedule=None,
start_date=dt.datetime(2023, 7, 25, 0, 0, 0, tzinfo=dt.timezone.utc),
)
def test_dag():
@task(task_id='test_task', templates_dict={'query': 'sql/myquery.sql'}, templates_exts=['.sql'])
def test_task(templates_dict=None):
print(templates_dict['query'])
@task.short_circuit(task_id='test_tasksc', templates_dict={'query': 'sql/myquery.sql'}, templates_exts=['.sql'])
def test_tasksc(templates_dict=None):
print(templates_dict['query'])
return True
test_task() >> test_tasksc()
test_dag()
Output of first task: SELECT * FROM ....
Output of second task: sql/myquery.sql.
As a guess, I think the problem could be in this line, where templates_dict and templatex_ext are not explicitly passed to the super-class's init function. I am happy to make an MR if it's that small of a change.
What you think should happen instead
Output of first task: SELECT * FROM ....
Output of second task: SELECT * FROM ....
How to reproduce
See MRE above.
Operating System
RHEL 7.9 (Maipo)
Versions of Apache Airflow Providers
apache-airflow==2.5.0
No relevant providers.
Deployment
Virtualenv installation
Deployment details
No response
Anything else
No response
Are you willing to submit PR?
Code of Conduct
Apache Airflow version
Other Airflow 2 version (please specify below)
What happened
I believe that the
@task.short_circuitoperator is missing the code to handle the usage oftemplates_dict, as described in the documentation. The code snippet below demonstrates the issue. In it, I create two tasks. The first is a basic python task, and in the log, from the print statement, I see the contents of the.sqlfile. (So the templating worked correctly.) However, in the log for the second task, I see only the string itself, with no jinja templating performed. I think this contradicts the documentation.Output of first task:
SELECT * FROM ....Output of second task:
sql/myquery.sql.As a guess, I think the problem could be in this line, where
templates_dictandtemplatex_extare not explicitly passed to the super-class's init function. I am happy to make an MR if it's that small of a change.What you think should happen instead
Output of first task:
SELECT * FROM ....Output of second task:
SELECT * FROM ....How to reproduce
See MRE above.
Operating System
RHEL 7.9 (Maipo)
Versions of Apache Airflow Providers
apache-airflow==2.5.0
No relevant providers.
Deployment
Virtualenv installation
Deployment details
No response
Anything else
No response
Are you willing to submit PR?
Code of Conduct