-
Notifications
You must be signed in to change notification settings - Fork 16.3k
Closed
Labels
Description
Apache Airflow version
2.2.2
What happened
Introduced here 0a2d0d1 databricks operators now define template_ext property as ('.json',). This change broke a few dags we have currently as they basically define a config json file that needs to be posted to databricks. Example:
DatabricksRunNowOperator(
task_id=...,
job_name=...,
python_params=["app.py", "--config", "/path/to/config/inside-docker-image.json"],
databricks_conn_id=...,
email_on_failure=...,
)This snippet will make airflow to load /path/to/config/inside-docker-image.json and it is not desired.
@utkarsharma2 @potiuk can this change be reverted, please? It's causing headaches when a json file is provided as part of the dag parameters.
What you think should happen instead
Use a more specific extension for databricks operators, like .json-tpl
How to reproduce
No response
Operating System
Any
Versions of Apache Airflow Providers
apache-airflow-providers-databricks==2.6.0
Deployment
Virtualenv installation
Deployment details
No response
Anything else
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
JoseLMoreira, douglaspenna, ffdavidecruz and edc-ffdouglaspenna and JoseLMoreira