You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I see airflow.operators.ExternalTaskSensor supports dependency on an ask in another DAG. But is it possible to set dependency on another DAG itself? In other words, can external_task_id parameter be optional?
Not for now but it'd be easy to either write an ExternalDagSensor, or add a DummyOperator to the DAG you want to depend on that would be downstream of all the tasks, and use the ExternalTaskSensor against it.
@mistercrunch sorry to hijack this thread but I've been using the DummyOperator approach for a couple Dags and was considering writing an ExternalDagSensor. Would you accept a PR for one? My thought was it would be identical to ExternalTaskSensor except instead of looking at just one task it would look at every task in the DAG (which it would retrieve from the ORM).
@jlowin , ExternalDagSensor sounds good, check out the DAG's root property to get all tasks without downstream deps.
I like for everything in a DAG to converge to a last point (an email operator + slack operator that post about completion) but I can understand that not everyone build their DAGs that way.
Activity
mistercrunch commentedon Jun 5, 2015
Not for now but it'd be easy to either write an
ExternalDagSensor, or add aDummyOperatorto the DAG you want to depend on that would be downstream of all the tasks, and use theExternalTaskSensoragainst it.dirkgently007 commentedon Jun 11, 2015
Thanks @mistercrunch . I think
DummyOperatorshould be an easy solution.Should I close this?
mistercrunch commentedon Jun 11, 2015
Yup, closing.
jlowin commentedon Sep 29, 2015
@mistercrunch sorry to hijack this thread but I've been using the
DummyOperatorapproach for a couple Dags and was considering writing anExternalDagSensor. Would you accept a PR for one? My thought was it would be identical toExternalTaskSensorexcept instead of looking at just one task it would look at every task in the DAG (which it would retrieve from the ORM).mistercrunch commentedon Oct 1, 2015
@jlowin ,
ExternalDagSensorsounds good, check out the DAG'srootproperty to get all tasks without downstream deps.I like for everything in a DAG to converge to a last point (an email operator + slack operator that post about completion) but I can understand that not everyone build their DAGs that way.
fb64f2e: [TWTR][AIRFLOW-XXX] Twitter Airflow Customizations + Fixup j…
bump airflow version (apache#6)
Merge pull request apache#6 from OpenLineage/julienledem-talks
# This is a combination of 7 commits.
chore(deps): update dependency husky to v3.0.3 (apache#6)