Skip to content

I need a flow to start with a sensor #57

@r39132

Description

@r39132

I have a flow that is linear and sequential as shown below :

  1. new data arrives in SQS
  2. wait for that data to start being written to postgres
  3. wait for the SQS queue to drain
  4. send an email that the flow completed

Eventually, we will have a pre-cursor step (e.g. step 0.) that kicks off a Spark job before waiting for new data in the SQS queue. For now, I do not have that, so I would like my flow to simply reset to the first step. In some ways, I am asking for a circular flow that never ends. This breaks the entire workflow abstraction. On another note, where is the "cron" functionality? Do I schedule airflow commands in cron outside of airflow?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions