Skip to content

[C++] Casting a string to a decimal type wraps values wider than the storage integer instead of erroring #55606

Open

Description

@pedrobtz

Environment

  • pyarrow 26.0.1 (PyPI wheel), and the arrow R package 26.0.1 (apache.r-universe.dev binary) and 26.0.0 (CRAN binary)
  • macOS 15 (Darwin 24.6.0), x86_64

Description

A string-to-decimal cast checks the parsed value against the target precision and errors when it does not fit. That check holds while the value fits in the storage integer (32, 64, 128 or 256 bits). A value wider than the storage integer is reduced modulo 2^width first. The precision check then sees the wrapped value, and the cast returns it with no error.

Smaller out-of-range values error as expected: "123456" cast to decimal128(5, 0) gives "Decimal value does not fit in precision 5".

Minimal reproducible example

import pyarrow as pa, pyarrow.compute as pc

cases = [("9" * 39,       pa.decimal128(38, 0)),
         (str(2**64),     pa.decimal64(18, 0)),
         (str(2**256 + 7), pa.decimal256(76, 0)),
         ("9" * 38,       pa.decimal128(38, 0)),   # fits: control
         ("123456",       pa.decimal128(5, 0))]    # too wide for precision: control
for s, t in cases:
    try:
        r = pc.cast(pa.array([s]), t)[0].as_py()
    except Exception as e:
        r = "ERROR: %s" % e
    print(s[:10] + "...", "->", t, ":", r)

The R bindings behave the same: Array$create(strrep("9", 39))$cast(decimal128(38, 0)).

Observed

9999999999... -> decimal128(38, 0) : -20847100762815390390123822295304634369
1844674407... -> decimal64(18, 0) : 0
1157920892... -> decimal256(76, 0) : 7
9999999999... -> decimal128(38, 0) : 99999999999999999999999999999999999999
123456... -> decimal128(5, 0) : ERROR: Decimal value does not fit in precision 5

Expected

The first three casts should fail with the same "does not fit in precision" error as the last line: 39 nines need precision 39, 2^64 needs 20 and 2^256 + 7 needs 78. A cast should never return a different number than the one it parsed, and a negative value from a positive input is a clear sign of overflow.

Likely cause

The digit accumulation in the decimal string parser (Decimal{32,64,128,256}::FromString and the cast kernel that uses it) multiplies and adds in the storage integer without checking for overflow. Then FitsInPrecision() runs on the wrapped result. Counting the significant digits before accumulating, or checking overflow at each step, would catch it.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions