Apache Airflow version
Other Airflow 2 version (please specify below)
What happened
Hello ,
we are facing "504 Gateway Time-out" error when running "tasks run -m" CLI command. We are trying to create Complex DAG and run tasks from cli command. When we are trying to run "tasks run -m" then we received gateway timeout error.
We also observed high resources spike in web server when we tried to execute this cli command. After looking into further, when we run "task run -m" Airflow CLI command, what it does is it parses the list of DAGs and the parses through the task list. Because this , we observed high resources of webserver and received gateway timeout error.
What you think should happen instead
We are expecting that when to execute "tasks run" cli command, it should only parse DAG name and task name provided in the command and not parse DAG lists followed by task list.
How to reproduce
please follow below steps to reproduce this issue.
- we have 900 DAG in airflow environment.
- we have created web login token to access web server.
- after that we tried to run "tasks run" using python script
Operating System
Amazon linux
Versions of Apache Airflow Providers
Airflow version 2.2.2
Deployment
Amazon (AWS) MWAA
Deployment details
No response
Anything else
No response
Are you willing to submit PR?
Code of Conduct
Apache Airflow version
Other Airflow 2 version (please specify below)
What happened
Hello ,
we are facing "504 Gateway Time-out" error when running "tasks run -m" CLI command. We are trying to create Complex DAG and run tasks from cli command. When we are trying to run "tasks run -m" then we received gateway timeout error.
We also observed high resources spike in web server when we tried to execute this cli command. After looking into further, when we run "task run -m" Airflow CLI command, what it does is it parses the list of DAGs and the parses through the task list. Because this , we observed high resources of webserver and received gateway timeout error.
What you think should happen instead
We are expecting that when to execute "tasks run" cli command, it should only parse DAG name and task name provided in the command and not parse DAG lists followed by task list.
How to reproduce
please follow below steps to reproduce this issue.
Operating System
Amazon linux
Versions of Apache Airflow Providers
Airflow version 2.2.2
Deployment
Amazon (AWS) MWAA
Deployment details
No response
Anything else
No response
Are you willing to submit PR?
Code of Conduct