Integrate BatchCoalescer into AsyncFuncExec and remove from CoalesceBatches optimization rule #19331
Description
Is your feature request related to a problem or challenge?
The LimitedBatchCoalescer can now be used to integrate batch coalescing directly in ExecutionPlan implementations, which is better than using an external operator for the reasons described on #18779
Currently AsyncFuncExec needs a CoalesceBatches operator after for performance, see
We should integrate coalescing directly in the operator
Describe the solution you'd like
- Add batch coalescing into
AsyncFuncExec - Remove
AsyncFuncExecfrom theCoalesceBatchesoptimizer rule:
Describe alternatives you've considered
@jizezhang did this for RepartitionExec here:
@Dandandan did this for FilterExec in these two PRs, so we can probably follow a similar model
- Coalesce batches inside FilterExec #18604
- Remove FilterExec from CoalesceBatches optimization rule, add fetch support #18630
Additional context
No response
Reacted by feniljain
Activity
feniljain commentedon Dec 15, 2025
take
alamb commentedon Dec 16, 2025
BTW I think @@jizezhang may also be working on this
jizezhang commentedon Dec 16, 2025
I was about to, but I saw @feniljain took it. Please feel free to continue @feniljain . and thanks for mentioning @alamb :)
alamb commentedon Dec 16, 2025
Perhaps you would be willing to help review the PR @jizezhang ? It looks like @feniljain has started here
jizezhang commentedon Dec 17, 2025
Yes of course, looks like the PR is in draft currently, will take a look when it's ready.
feniljain commentedon Dec 17, 2025
Hey folks! 👋🏻
Apologies I didn't notice @jizezhang was also working on it, I didn't see a linked PR/assignee to the ticket and started working on it, maybe I am missing a conversation somewhere 😅
Thanks for your consideration @jizezhang 🙇🏻
Will make sure to tag you as I get the PR ready :)
feat: integrate batch coalescer with async fn exec (#19342)