Skip to content

Can you have a DAG dependent on another DAG? #6

Closed

Description

@dirkgently007

I see airflow.operators.ExternalTaskSensor supports dependency on an ask in another DAG. But is it possible to set dependency on another DAG itself? In other words, can external_task_id parameter be optional?

Activity

  1. mistercrunch commented on Jun 5, 2015

    @mistercrunch
    Member

    Not for now but it'd be easy to either write an ExternalDagSensor, or add a DummyOperator to the DAG you want to depend on that would be downstream of all the tasks, and use the ExternalTaskSensor against it.

  2. dirkgently007 commented on Jun 11, 2015

    @dirkgently007
    Author

    Thanks @mistercrunch . I think DummyOperator should be an easy solution.

    Should I close this?

  3. mistercrunch commented on Jun 11, 2015

    @mistercrunch
    Member

    Yup, closing.

  4. jlowin commented on Sep 29, 2015

    @jlowin
    Member

    @mistercrunch sorry to hijack this thread but I've been using the DummyOperator approach for a couple Dags and was considering writing an ExternalDagSensor. Would you accept a PR for one? My thought was it would be identical to ExternalTaskSensor except instead of looking at just one task it would look at every task in the DAG (which it would retrieve from the ORM).

  5. mistercrunch commented on Oct 1, 2015

    @mistercrunch
    Member

    @jlowin , ExternalDagSensor sounds good, check out the DAG's root property to get all tasks without downstream deps.

    I like for everything in a DAG to converge to a last point (an email operator + slack operator that post about completion) but I can understand that not everyone build their DAGs that way.

  6. added a commit that references this issue on Apr 10, 2020
  7. added 2 commits that reference this issue on Jan 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions