Skip to content

Document DataFusion optimizer rules #21771

Closed

Description

@comphead

Is your feature request related to a problem or challenge?

Existing docs:

  • docs/source/library-user-guide/query-optimizer.md — good high-level overview of the optimizer architecture + how to write custom rules. Does NOT list/describe individual built-in rules.
  • datafusion/optimizer/README.md and datafusion/physical-optimizer/README.md — brief crate-level READMEs, mostly pointers to the above doc and blog posts.
  • working-with-exprs.md and extending-operators.md — touch on optimizer rules tangentially.

The gap: There is no reference doc that lists and describes what each built-in rule actually does. The codebase has 27 logical optimizer rules (e.g., push_down_filter, eliminate_cross_join,
common_subexpr_eliminate) and 21 physical optimizer rules (e.g., join_selection, enforce_sorting, topk_aggregation), but the only way to learn what each one does is to read the source or use EXPLAIN
VERBOSE.

So query-optimizer.md is good for "how the optimizer works" and "how to extend it," but it's not enough if you want a reference of "what optimizations does DataFusion apply and what does each one do."
That reference doesn't exist yet.

Describe the solution you'd like

No response

Describe alternatives you've considered

No response

Additional context

No response

Activity

  1. added theissue type on Apr 22, 2026
  2. Adez017 commented on Apr 24, 2026

    @Adez017
    Contributor

    take

  3. Adez017 commented on Apr 24, 2026

    @Adez017
    Contributor

    Hi @comphead , i would like to work upon this one . perhaps will need some assistance if required .

  4. kumarUjjawal commented on Apr 24, 2026

    @kumarUjjawal
    Contributor

    Hi @comphead , i would like to work upon this one . perhaps will need some assistance if required .

    Hi @Adez017 I have already been working on this and have opened the pr here #21824 maybe you can help review it.

  5. added a commit that references this issue on Apr 29, 2026
    f0c5306
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationenhancementNew feature or requestoptimizerOptimizer rules

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions