Skip to content

Create simple "Task API" server based on FastAPI #43009

Closed

Description

@kaxil

Need to work with Pierre on this to work out the right code and URI layout for this.

I think there have been some people who want to run them separately (it was certainly asked about by someone in my talk), so I think they should be structured as two separate FastAPI/ASGI apps that can optionally be run in a single binary (i.e. ASGI "mounting" etc.). At this stage I'm thinking that we should not separate out the dependencies/make them stand-alone projects (that can happen later if there proves to be enough desire)

Activity

  1. self-assigned this
    on Oct 14, 2024
  2. converted this from a draft issue on Oct 14, 2024
  3. added this to the Airflow 3.0.0 milestone on Oct 14, 2024
  4. potiuk commented on Oct 15, 2024

    @potiuk
    Member

    I think it really depends on how we want to treat providers - this is the big difference of dependencies. Technically - if we solve base links and connection issue, we could have the same dependencies for both - none of the task api calls nor web API should need providers at all in this case. On the other hand, if we don't solve it, then both will have providers if they are run together - but task API technically could be separate and have no provider dependencies.

    I think we should aim on having the same dependencies in both for simplicity, and we should see if we can remove providers from both - but this can also be done later - either before 3.0 or after - this only changes security property of the deployment, so it won't be a breaking change.

  5. kaxil commented on Oct 15, 2024

    @kaxil
    MemberAuthor

    Yea, we absolutely need to solve Operator links & Connection deps. Jens & Vikram have a proposal. One of the last dev calls (Sep 19) we discussed about routing it through an API (and stored it in DB):

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions