Skip to content

New grid view in Airflow 2.3.0 has very slow performance on large DAGs relative to tree view in 2.2.5 #23772

@joshzana

Description

@joshzana

Apache Airflow version

2.3.0 (latest released)

What happened

I upgraded a local dev deployment of Airflow from 2.2.5 to 2.3.0, then loaded the new /dags/<dag_id>/grid page for a few dag ids.

On a big DAG, I’m seeing 30+ second latency on the /grid API, followed by a 10+ second delay each time I click a green rectangle. For a smaller DAG I tried, the page was pretty snappy.

I went back to 2.2.5 and loaded the tree view for comparison, and saw that the /tree/ endpoint on the large DAG had 9 seconds of latency, and clicking a green rectangle had instant responsiveness.

This is slow enough that it would be a blocker for my team to upgrade.

What you think should happen instead

The grid view should be equally performant to the tree view it replaces

How to reproduce

Generate a large DAG. Mine looks like the following:

  • 900 tasks
  • 150 task groups
  • 25 historical runs

Compare against a small DAG, in my case:

  • 200 tasks
  • 36 task groups
  • 25 historical runs

The large DAG is unusable, the small DAG is usable.

Operating System

Ubuntu 20.04.3 LTS (Focal Fossa)

Versions of Apache Airflow Providers

No response

Deployment

Docker-Compose

Deployment details

Docker-compose deployment on an EC2 instance running ubuntu.
Airflow web server is nearly stock image from apache/airflow:2.3.0-python3.9

Anything else

Screenshot of load time:
image

GIF of click latency:
2022-05-17 21 26 26

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Labels

area:UIRelated to UI/UX. For Frontend Developers.area:webserverWebserver related Issueskind:bugThis is a clearly a bug

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions