Skip to content

Consider optimizing LeftSemi to emit output incrementally #22929

Description

@neilconway

Is your feature request related to a problem or challenge?

Right now, LeftSemi consumes the entire probe side before emitting any output rows. This isn't fundamental: we could check which probe row flips the visited bitmap from 0 -> 1, and arrange for the output row to be produced at that point. This would enable better pipeline parallelism.

It would probably also improve partitioning behavior: LeftSemi + CollectLeft emits everything in a single output partition today, which isn't great.

Would need to check if this needs stronger memory ordering requirements when flipping bits than we can get away with today.

Describe the solution you'd like

No response

Describe alternatives you've considered

No response

Additional context

No response

Activity

  1. neilconway commented on Jun 12, 2026

    @neilconway
    ContributorAuthor

    take

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions