Body

There is a task context element outlet_events that deals with assets but not events.

So, the task isn't currently able to modify the actual events that get created -- rather it can just manipulate an asset object (which doesn't have all the same properties as asset events do).

This means that with the current design there's no way for the task to say what partition was touched for example.

Is this ok? Dunno. It's certainly not ideal. But we need to think through what if anything we shall do about it. Should we modify / extend this interface somehow? Or add a new interface that really deals with the events?

Currently we're just naively taking the partition key from the dag run. But, it's definitely conceivable that users may not want all assets in all tasks to have the partition from that dag run.

So maybe we extend "outlet_events" so that users could optionally put asset events there (instead of the asset) and then, if an event is there then we respect it.

Then we'd also want some way to say "no event" -- how would we say that?

Maybe if the asset key has a value of None

And it's conceivable that a user would want to put a list of asset events for the asset and not just one. E.g. if the task for some reason chose to update three partitions for the same asset.

link to todo in code:

@uranusjr @Lee-W

Committer

• I acknowledge that I am a maintainer/committer of the Apache Airflow project.