Skip to content

Airflow tasks run -m cli command giving 504 response #32020

Description

@vidyapanchalZS

Apache Airflow version

Other Airflow 2 version (please specify below)

What happened

Hello ,

we are facing "504 Gateway Time-out" error when running "tasks run -m" CLI command. We are trying to create Complex DAG and run tasks from cli command. When we are trying to run "tasks run -m" then we received gateway timeout error.

We also observed high resources spike in web server when we tried to execute this cli command. After looking into further, when we run "task run -m" Airflow CLI command, what it does is it parses the list of DAGs and the parses through the task list. Because this , we observed high resources of webserver and received gateway timeout error.

What you think should happen instead

We are expecting that when to execute "tasks run" cli command, it should only parse DAG name and task name provided in the command and not parse DAG lists followed by task list.

How to reproduce

please follow below steps to reproduce this issue.

  1. we have 900 DAG in airflow environment.
  2. we have created web login token to access web server.
  3. after that we tried to run "tasks run" using python script

Operating System

Amazon linux

Versions of Apache Airflow Providers

Airflow version 2.2.2

Deployment

Amazon (AWS) MWAA

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions