Skip to content

Writing a Timestamp in the last ~26h of year 9999 panics during Min/Max stats #8860

Closed

Description

@tobias-fire

What happens

When a Timestamp column holds a value greater than jiff's Timestamp maximum (253402207200 s = 9999-12-30T22:00:00Z), VortexWriteOptions::write panics (via vortex_panic!) while computing the Min/Max statistic. Vortex builds an extension Scalar for the min/max value and validates it through jiff; for values in the last ~26 hours of year 9999 that validation fails and surfaces as a panic rather than an error. Arrow, Parquet, and many SQL engines allow timestamps up to 9999-12-31T23:59:59, so this can be triggered by ordinary in-range data.

Version: vortex 0.78.0.

Minimal repro (only depends on vortex = "=0.78.0"):

use std::sync::LazyLock;
use vortex::VortexSessionDefault;
use vortex::array::arrays::{ExtensionArray, StructArray};
use vortex::array::extension::datetime::{TimeUnit, Timestamp};
use vortex::array::{ArrayRef, IntoArray};
use vortex::buffer::{ByteBufferMut, buffer};
use vortex::dtype::Nullability;
use vortex::file::WriteOptionsSessionExt;
use vortex::io::runtime::BlockingRuntime;
use vortex::io::runtime::current::CurrentThreadRuntime;
use vortex::io::session::RuntimeSessionExt;
use vortex::session::VortexSession;

static RUNTIME: LazyLock<CurrentThreadRuntime> = LazyLock::new(CurrentThreadRuntime::new);
static SESSION: LazyLock<VortexSession> =
    LazyLock::new(|| VortexSession::default().with_handle(RUNTIME.handle()));

fn write_one_timestamp(ts_micros: i64) -> usize {
    let ext = Timestamp::new(TimeUnit::Microseconds, Nullability::NonNullable).erased();
    let storage: ArrayRef = buffer![ts_micros].into_array();
    let ts: ArrayRef = ExtensionArray::new(ext, storage).into_array();
    let table = StructArray::from_fields(&[("t", ts)]).unwrap().into_array();
    let mut buf = ByteBufferMut::empty();
    RUNTIME.block_on(async { SESSION.write_options().write(&mut buf, table.to_array_stream()).await })
        .expect("write should succeed");
    buf.len()
}

fn main() {
    const IN_RANGE:   i64 = 253_402_207_200_000_000; // 9999-12-30T22:00:00Z (jiff max) — OK
    const OVER_RANGE: i64 = 253_402_214_400_000_000; // 9999-12-31T00:00:00Z — panics
    println!("in-range: wrote {} bytes", write_one_timestamp(IN_RANGE));
    println!("over-range: wrote {} bytes", write_one_timestamp(OVER_RANGE)); // panics here
}

Output:

in-range  (9999-12-30 22:00:00Z): wrote 2676 bytes
over-range (9999-12-31 00:00:00Z): writing...
thread 'main' panicked at vortex-error-0.78.0/src/lib.rs:347:
unable to construct an extension `Scalar`:
  Other error: Invalid timestamp scalar: parameter 'Unix timestamp seconds' is not in the required range of -377705023201..=253402207200

The value at the limit (9999-12-30T22:00:00Z) writes fine; the same column two hours later aborts, which points to the jiff range boundary.

Root cause

  • aggregate_fn/fns/min_max/extension.rs::accumulate_extension computes min/max on the i64 storage, then wraps the results with the infallible Scalar::extension_ref (internally Scalar::try_new(...).vortex_expect("unable to construct an extension Scalar")).
  • extension/datetime/timestamp.rs::unpack_native validates via jiff::Timestamp::UNIX_EPOCH.checked_add(span), and jiff intentionally reserves headroom (so a Timestamp stays convertible to a civil datetime in any timezone), so it can't represent the final ~26h of year 9999.
  • The fallible result becomes a panic, which aborts the writer — in our case across a C FFI boundary, so it takes down the whole process.

Expected behavior

Ideally, support the full timestamp range that Arrow and Parquet allow (through the end of 9999-12-31). Alternatively, skip the Min/Max statistic when a value can't be represented as an extension scalar — the data itself round-trips fine, so dropping the stat for that column/chunk is preferable to a panic.

Activity

  1. self-assigned this
    on Aug 11, 2026
  2. robert3005 commented on Aug 11, 2026

    @robert3005
    Contributor

    BurntSushi/jiff#302. I think BurntSushi tradeoff here is sensible. We definitely shouldn't error... sadly we need to support the other sql behaviours

  3. fredlarochelle commented on Oct 1, 2026

    @fredlarochelle

    Accidentally caught a related panic while writing a test, one that making min/max's scalar construction fallible alone wouldn't fix.

    Timestamp::unpack_native builds its jiff::Span using panicking setters, so values outside the span field's range panic inside Jiff before reaching checked_add. TimestampValue's Display implementation also use these setters.

    On 0.86.1, adapting your repro to use TimeUnit::Nanoseconds and i64::MIN panics with value for nanoseconds is out of bounds ; i64::MIN + 1 writes fine.

    jiff::Timestamp::from_nanosecond(i128::from(value)) would avoid it, though it would not resolve the year-9999 range limitation from the original report.

  4. added a commit that references this issue on Nov 3, 2026
    8896ab2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugA bug issue

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions