Skip to content

chore(core): support for Decimal arithmetic#5991

Merged
bluestreak01 merged 97 commits intomasterfrom
vi_decimal
Aug 18, 2025
Merged

chore(core): support for Decimal arithmetic#5991
bluestreak01 merged 97 commits intomasterfrom
vi_decimal

Conversation

@bluestreak01
Copy link
Copy Markdown
Member

@bluestreak01 bluestreak01 commented Jul 28, 2025

TODO:

  • Arithmetic operators with overflow
  • Fuzzy testing
  • Decimal64
  • Decimal128
  • Decimal256

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Jul 31, 2025

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

Introduces new fixed-point decimal types Decimal64, Decimal128, and Decimal256 with arithmetic, scaling, conversion, and formatting APIs; a Knuth-based divider; extensive JMH benchmarks for add/subtract/multiply/divide/round across 128/256-bit types vs BigDecimal; random decimal generators in Rnd; a redesigned NumericException; a small interface addition; and a minor Maven config tweak.

Changes

Cohort / File(s) Summary
Benchmarks: Decimal128
benchmarks/src/main/java/org/questdb/Decimal128AddBenchmark.java, .../Decimal128SubtractBenchmark.java, .../Decimal128MultiplyBenchmark.java, .../Decimal128DivideBenchmark.java, .../Decimal128RoundBenchmark.java
Adds JMH benchmarks for Decimal128 vs BigDecimal covering add, subtract, multiply, divide, and round with multiple scenarios and runners.
Benchmarks: Decimal256
benchmarks/src/main/java/org/questdb/Decimal256AddBenchmark.java, .../Decimal256SubtractBenchmark.java, .../Decimal256MultiplyBenchmark.java, .../Decimal256DivideBenchmark.java, .../Decimal256RoundBenchmark.java
Adds JMH benchmarks for Decimal256 vs BigDecimal covering add, subtract, multiply, divide, and round with scenario-driven setups and runners.
Decimal numeric types
core/src/main/java/io/questdb/std/Decimal64.java, .../Decimal128.java, .../Decimal256.java, .../DecimalKnuthDivider.java
Introduces mutable fixed-scale decimal implementations (64/128/256-bit) with in-place and static arithmetic, scaling, conversion, formatting; adds a Knuth divider for high-precision division and rounding.
Random generators
core/src/main/java/io/questdb/std/Rnd.java
Adds random value generators for Decimal64/128/256 with sink and factory methods.
Exception redesign
core/src/main/java/io/questdb/std/NumericException.java
Refactors to RuntimeException with flyweight semantics, thread-local instance, sinkable message building, and chaining put(...) APIs; deprecates static INSTANCE usage.
Engine interface
core/src/main/java/io/questdb/griffin/engine/table/UpdateExistingInterface.java
Adds functional interface for updating existing entries with MapValue, Record, and rowId.
Tests
core/src/test/java/io/questdb/test/std/Decimal64Test.java, .../Decimal128Test.java, .../Decimal256Test.java, .../NumericExceptionTest.java
Adds comprehensive unit tests for decimal types and NumericException, including fuzzing, rounding, conversions, overflow/edge cases, and formatting.
Build config
core/pom.xml
Removes Surefire true setting.

Sequence Diagram(s)

sequenceDiagram
  participant JMH as JMH Runner
  participant BM as *Decimal256DivideBenchmark*
  participant D256 as Decimal256
  participant KD as DecimalKnuthDivider

  JMH->>BM: invoke decimal256Divide()
  BM->>D256: copyFrom(dividend)
  BM->>D256: divide(divisor, scale, roundingMode)
  D256->>KD: instance().ofDividend(...).ofDivisor(...).divide(...)
  KD-->>D256: sink(quotient, scale)
  D256-->>BM: result
  BM-->>JMH: return result
Loading
sequenceDiagram
  participant App as Calling code
  participant NE as NumericException
  participant Sink as CharSink

  App->>NE: NumericException.instance()
  App->>NE: position(0).put("Error: ").put(42).put(' ')
  App->>Sink: new StringSink()
  NE->>Sink: toSink(sink)
  Sink-->>App: "Error: 42 "
Loading

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~90 minutes

✨ Finishing Touches
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch vi_decimal

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

‼️ IMPORTANT
Auto-reply has been disabled for this repository in the CodeRabbit settings. The CodeRabbit bot will not respond to your replies unless it is explicitly tagged.

  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Status, Documentation and Community

  • Visit our Status Page to check the current availability of CodeRabbit.
  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@nwoolmer nwoolmer mentioned this pull request Jul 31, 2025
1 task
@RaphDal RaphDal changed the title chore(core): support for Decimal128 arithmetic chore(core): support for Decimal160 arithmetic Aug 4, 2025
@nmarasoiu
Copy link
Copy Markdown

nmarasoiu commented Aug 4, 2025

Hi, where can I ask questions at the global level? #1565 ? #4504 ? questdb/roadmap#89 ? Or another, or is it ok if i ask here?

I believe your thinking might be the same, but wanted to say hi and double check.

Q: in the task above called "automatic scale", do you mean
a. having the Decimal160 work with a parametric compatible scale, as opposed to current efficient but hardcoded scale? (algo wise; independent on where scale is stored),
b. or do you consider encoding the scale in the value itself, instead of in the column type?

What I mean to ask is are you considering math ops between numbers with different scale?

I'm thinking that potential disadvantages to scale encoded in each value (vs encoded in column type), at bit lengths from 32 to 256, include

  • potential math speed & complexity drawbacks between numbers of potentially different scale
  • some loss of precision due to storing scale metadata (though at 160 bits & scale <= 32, this loss is minimal)

isn't it? But you would know better, pls lemme know.

Especially since for money in particular (the origin of the business request i saw, may be missing other use cases), they would likely be ok with scale defined at column level, isnt' it?

eg MONEY == DECIMAL38(22) where 38 = precision and 20 = scale,
so sign + 10^18 the integer part (1 trillion USD in Iranian RIAL) + 20 after decimal point, where

on int:     DECIMAL9(4)   → sign + 5 digits before, 4 after decimal point
on long:    DECIMAL19(4)  → sign + 15 digits before, 4 after decimal point
on long128: DECIMAL38(s) 
on long160: DECIMAL47(s)
on long256: DECIMAL76(s) 

or we could use the postgres DECIMAL(p, s) but only accept precision <= 76, what do you think?

@nmarasoiu
Copy link
Copy Markdown

Also it was probably considered but double double ie double128 or double160 could be an alternative? Though the encoding scale in value is a drawback.

Alternatively could we emit such types of different widths and scale when fixed for a type as bytecode at runtime like we do for JIT?

@questdb questdb deleted a comment from coderabbitai bot Aug 16, 2025
RaphDal and others added 19 commits August 16, 2025 18:13
@RaphDal RaphDal self-requested a review August 18, 2025 14:16
@glasstiger
Copy link
Copy Markdown
Contributor

[PR Coverage check]

😍 pass : 1927 / 1934 (99.64%)

file detail

path covered line new line coverage
🔵 io/questdb/std/DecimalKnuthDivider.java 196 199 98.49%
🔵 io/questdb/std/Decimal256.java 1044 1046 99.81%
🔵 io/questdb/std/Decimal64.java 215 217 99.08%
🔵 io/questdb/std/Decimal128.java 393 393 100.00%
🔵 io/questdb/std/NumericException.java 33 33 100.00%
🔵 io/questdb/std/Rnd.java 46 46 100.00%

@bluestreak01 bluestreak01 merged commit 330ac8b into master Aug 18, 2025
35 checks passed
@bluestreak01 bluestreak01 deleted the vi_decimal branch August 18, 2025 15:07
@RaphDal RaphDal mentioned this pull request Aug 19, 2025
57 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants