Skip to content
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add back link
  • Loading branch information
alamb committed Mar 7, 2025
commit 05037125e946d3c6554632421fdc429d03a2a623
4 changes: 3 additions & 1 deletion datafusion/optimizer/src/decorrelate.rs
Original file line number Diff line number Diff line change
Expand Up @@ -56,14 +56,16 @@ pub struct PullUpCorrelatedExpr {
/// Indicates if we encounter any correlated expression that can not be pulled up
/// above a aggregation without changing the meaning of the query.
can_pull_over_aggregation: bool,
/// Do we need to handle the [count bug] during the pull up process.
/// Do we need to handle [the count bug] during the pull up process.
///
/// The "count bug" was described in [Optimization of Nested SQL

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @suibianwanwank I'm feeling still having an original GH link would be beneficial ?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I put it back in 0503712

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @comphead @alamb I also made a similar change in with_need_handle_count_bug.

/// Queries Revisited](https://dl.acm.org/doi/pdf/10.1145/38714.38723). This bug is
/// not specific to the COUNT function, and it can occur with any aggregate function,
/// such as SUM, AVG, etc. The anomaly arises because aggregates fail to distinguish
/// between an empty set and null values when optimizing a correlated query as a join.
/// Here, we use "the count bug" to refer to all such cases.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @suibianwanwank -- this is great! I learned something new!

///
/// [the count bug]: https://github.com/apache/datafusion/issues/10553
pub need_handle_count_bug: bool,
/// mapping from the plan to its expressions' evaluation result on empty batch
pub collected_count_expr_map: HashMap<LogicalPlan, ExprResultMap>,
Expand Down