Skip to content

feat: Support ANSI mode SUM (Decimal types)#2826

Merged
andygrove merged 8 commits intoapache:mainfrom
coderfender:add_ansi_support_sum_decimal_types
Dec 5, 2025
Merged

feat: Support ANSI mode SUM (Decimal types)#2826
andygrove merged 8 commits intoapache:mainfrom
coderfender:add_ansi_support_sum_decimal_types

Conversation

@coderfender
Copy link
Contributor

@coderfender coderfender commented Nov 25, 2025 •
edited
Loading

Which issue does this PR close?

Closed #531 (Adds ANSI support for decimal inputs)

Rationale for this change

  1. Changes to add ANSI and Try eval mode support to Decimal Sums
  2. The code already has support to identify overflow and the changes are essentially passing in eval mode and raising an error when there is an overflow. (is_null , is_empty flags). The only update here is to consolidate the state variables to 2 (optional i128 and an is_null flag). Unlike int sum, spark leverages both the sum and is_empty state vars in all modes

What changes are included in this PR?

  1. Changes to SumDecimal
  2. CometSum update to support ANSI mode
  3. Unit tests (and fix existing tests)

How are these changes tested?

  1. Unit test on scala side

@coderfender coderfender changed the title support_ansi_sum_decimal_input feat: Support ANSI Decimal Input SUM Nov 25, 2025
@coderfender
Copy link
Contributor Author

coderfender commented Nov 25, 2025

@andygrove , Please kickoff CI whenever get a chance . This PR should essentially finish impl ANSI (along with #2600 ) for all numeric types

@coderfender coderfender changed the title feat: Support ANSI Decimal Input SUM feat: Support ANSI mode for SUM Decimal types Nov 25, 2025
@coderfender coderfender changed the title feat: Support ANSI mode for SUM Decimal types feat: Support ANSI mode SUM (Decimal types) Nov 25, 2025
@codecov-commenter
Copy link

codecov-commenter commented Nov 25, 2025 •
edited
Loading

Codecov Report

❌ Patch coverage is 60.00000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 59.17%. Comparing base (f09f8af) to head (ab437f5).
⚠️ Report is 730 commits behind head on main.

Files with missing lines Patch % Lines
...main/scala/org/apache/comet/serde/aggregates.scala 60.00% 0 Missing and 2 partials ⚠️
Additional details and impacted files 
@@             Coverage Diff              @@
##               main    #2826      +/-   ##
============================================
+ Coverage     56.12%   59.17%   +3.05%     
- Complexity      976     1478     +502     
============================================
  Files           119      167      +48     
  Lines         11743    15188    +3445     
  Branches       2251     2522     +271     
============================================
+ Hits           6591     8988    +2397     
- Misses         4012     4916     +904     
- Partials       1140     1284     +144

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@coderfender
Copy link
Contributor Author

coderfender commented Nov 26, 2025

Fixed test failures by converting state variables into option and is_empty as per spark semantics . @andygrove , Please kickoff CI whenever you get a chance.

@coderfender
Copy link
Contributor Author

coderfender commented Nov 29, 2025

Fixed scala 2.13 formatting failures

martin-g
martin-g approved these changes Dec 4, 2025
View reviewed changes
Copy link
Member

@martin-g martin-g left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@coderfender
Copy link
Contributor Author

coderfender commented Dec 4, 2025

Thank you for the approval @martin-g

andygrove
andygrove approved these changes Dec 5, 2025
View reviewed changes
Copy link
Member

@andygrove andygrove left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @coderfender. One nit - the tests shouldn't be setting CometConf.getExprAllowIncompatConfigKey(classOf[Sum]) -> "true" so would be good to remove those lines.

@andygrove
Copy link
Member

andygrove commented Dec 5, 2025

Thanks @coderfender. One nit - the tests shouldn't be setting CometConf.getExprAllowIncompatConfigKey(classOf[Sum]) -> "true" so would be good to remove those lines.

This can be done in a separate PR

@andygrove
Copy link
Member

andygrove commented Dec 5, 2025

Thanks for the review @martin-g

@andygrove andygrove merged commit fd0ab64 into apache:main Dec 5, 2025
113 checks passed
@coderfender coderfender deleted the add_ansi_support_sum_decimal_types branch December 12, 2025 06:16
@andygrove andygrove mentioned this pull request Jan 23, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@martin-g martin-g martin-g approved these changes

@andygrove andygrove andygrove approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Add ANSI support in SUM and AVG

4 participants

@coderfender @codecov-commenter @andygrove @martin-g