AIP-67 - Introduce team id to executor names and loader#44710
Conversation
o-nikolas
requested review from
XD-DENG,
ashb,
hussein-awala and
pierrejeambrun
as code owners
boring-cyborg
Bot
added
the
area:Executors-core
When multi team is eventually in place, the scheduler will need to initialize a set of executors for each team. This change is a first step towards that which updates the ExecutorName data model and the executor loader to be "team aware".
o-nikolas
force-pushed
the
multi_team/onikolas/multi_executors_loader
branch
from
250f26c to
30396d8
Compare
|
CC @potiuk |
There was a problem hiding this comment.
I assume that it is done as part of AIP-67 - I think that it's worth mentioning it in the title and contents, as well as creating a project for that so it will become easier to track.
Before merging, it would be nice to outline a roadmap (general guidelines) so it would be clearer to understand the broader context of this task :)
Yupp this is AIP-67. That's a fair call out, I'll update the description, title and add a tag.
As I said above, I'll work on this, but it has never before been a requirement of merging PRs. And I don't think it's wise to add that barrier to our contribution process. So I don't think it should block this PR's merge. |
o-nikolas
changed the title
That import was removed in apache#44839, but apache#44710 wasn't up-to-date with main so static checks there didn't fail. This simply adds it back.
I apologize if it was interpreted as a barrier/blocker - it wasn't my intention at all. The doc that you created is exactly what I was looking for, just to understand how it's going to work. |
When multi team (AIP-67) is eventually in place, the scheduler will need to initialize a set of executors for each team. This change is a first step towards that which updates the ExecutorName data model and the executor loader to be "team aware"
That import was removed in apache#44839, but apache#44710 wasn't up-to-date with main so static checks there didn't fail. This simply adds it back.
6 participants
When multi team (AIP-67) is eventually in place, the scheduler will need to initialize a set of executors for each team. This change is a first step towards that which updates the ExecutorName data model and the executor loader to be "team aware".
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.