Tasks in DAGs with depends_on_past or task_concurrency are not being scheduled #12659
Description
Apache Airflow version: 1.10.13
What happened:
After performing an upgrade to v1.10.13 we noticed that tasks in some of our DAGs were not be scheduled. After a bit of investigation we discovered that by commenting out 'depends_on_past': True the issue went away.
What you expected to happen:
We think the issue might have something to do with this which was introduced to 1.10.13
[AIRFLOW-3607] Only query DB once per DAG run for TriggerRuleDep (#4751)
How to reproduce it:
- Install Airflow v1.10.13 from pip
- Start webserver and scheduler
- Add the following code as a DAG
- Switch the DAG on in the UI.
from airflow import models
from airflow.operators.dummy_operator import DummyOperator
from datetime import datetime, timedelta
default_args = {
'owner': 'airflow',
'start_date': datetime(2018, 10, 31),
'depends_on_past': True,
'retries': 3,
'retry_delay': timedelta(minutes=5)
}
dag_name = 'my-test-dag'
with models.DAG(dag_name,
default_args=default_args,
schedule_interval='0 0 * * *',
catchup=False,
max_active_runs=5,
) as dag:
test = DummyOperator(
task_id='test'
)