Skip to content

feat(sql): add timestamp and long window functions#6130

Merged
bluestreak01 merged 18 commits intomasterfrom
vi_first_timestamp
Sep 15, 2025
Merged

feat(sql): add timestamp and long window functions#6130
bluestreak01 merged 18 commits intomasterfrom
vi_first_timestamp

Conversation

@bluestreak01
Copy link
Copy Markdown
Member

@bluestreak01 bluestreak01 commented Sep 11, 2025

fixes #6122

While the fix was easy, I realised that reproducer was trying to execute first_value(timestamp), which we unjustifiably do not have implementation for. So I got AI to derive other type functions from the existing double ones.

Added new windows functions:

  • first_value(timestamp)
  • first_value(long)
  • last_value(timestamp)
  • last_value(long)
  • max(timestamp)
  • max(long)
  • min(timestamp)
  • min(long)

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Sep 11, 2025

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

Adds multiple window-function factories (first_value, last_value, min, max) for LONG and TIMESTAMP types, introduces TimestampNullFunction, changes LongNullFunction.zeroValue to primitive long, and adjusts async group-by cleanup to free-and-clear function lists.

Changes

Cohort / File(s) Summary
Abstract window base updates
core/src/main/java/io/questdb/griffin/engine/functions/window/AbstractWindowFunctionFactory.java
Converted LongNullFunction.zeroValue from Long to primitive long; added static inner TimestampNullFunction implementing WindowTimestampFunction, returning/writing a primitive zero-value in getTimestamp/pass1.
First-value window (long, timestamp)
core/src/main/java/io/questdb/griffin/engine/functions/window/FirstValueLongWindowFunctionFactory.java, core/src/main/java/io/questdb/griffin/engine/functions/window/FirstValueTimestampWindowFunctionFactory.java
New factories implementing first_value for LONG and TIMESTAMP with many variants (partitioned/unpartitioned, ROWS/RANGE, ignore/respect nulls), per-partition maps, memory-backed ring buffers, and RANGE ordering validation.
Last-value window (long, timestamp)
core/src/main/java/io/questdb/griffin/engine/functions/window/LastValueLongWindowFunctionFactory.java, core/src/main/java/io/questdb/griffin/engine/functions/window/LastValueTimestampWindowFunctionFactory.java
New factories implementing last_value for LONG and TIMESTAMP across framing/partitioning/null-handling combinations, using map state, CARW buffers, and many specialized inner implementations.
Max/Min window (long, timestamp)
core/src/main/java/io/questdb/griffin/engine/functions/window/MaxLongWindowFunctionFactory.java, core/src/main/java/io/questdb/griffin/engine/functions/window/MaxTimestampWindowFunctionFactory.java, core/src/main/java/io/questdb/griffin/engine/functions/window/MinLongWindowFunctionFactory.java, core/src/main/java/io/questdb/griffin/engine/functions/window/MinTimestampWindowFunctionFactory.java
New max/min factories for LONG and TIMESTAMP with comparator-driven implementations, RANGE validation, per-partition maps, ring-buffer-backed frames, optional deque-based candidate tracking, and appropriate NullFunction returns for empty frames.
Async group-by cleanup
core/src/main/java/io/questdb/griffin/engine/table/AsyncGroupByNotKeyedRecordCursorFactory.java
In _close(), replaced Misc.freeObjList(groupByFunctions) with Misc.freeObjListAndClear(groupByFunctions) to free objects and clear the list to avoid retained null references.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor Compiler as SQL Compiler
  participant Factory as WindowFunctionFactory
  participant Ctx as WindowContext
  participant Impl as WindowFunctionImpl
  participant SPI as WindowSPI

  Compiler->>Factory: newInstance(args, positions, config, ctx)
  Factory->>Ctx: inspect partitioning, framing, order, bounds, nulls
  alt invalid RANGE ordering
    Factory-->>Compiler: throw SqlException
  else empty frame
    Factory-->>Compiler: return NullFunction (Long/Timestamp)
  else
    Factory-->>Compiler: return specialized Impl
  end

  loop pass1 (single or streaming)
    Compiler->>Impl: pass1(record, offset, SPI)
    Impl->>SPI: write/update per-partition map or memory buffer
  end

  opt second pass required
    loop pass2
      Compiler->>Impl: pass2(record, offset, SPI)
      Impl->>SPI: compute/emit result
    end
  end
Loading

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120 minutes

Possibly related PRs

Suggested reviewers

  • kafka1991
✨ Finishing touches
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch vi_first_timestamp

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@bluestreak01
Copy link
Copy Markdown
Member Author

@CodeRabbit review

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Sep 11, 2025

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 8

🧹 Nitpick comments (19)
core/src/main/java/io/questdb/griffin/engine/functions/window/AbstractWindowFunctionFactory.java (1)

103-109: Consider clearing partitionByRecord functions on close.

To mirror the ObjList hygiene fix elsewhere and reduce chances of iterating nulls post-close, prefer freeObjListAndClear(...).

-                Misc.freeObjList(partitionByRecord.getFunctions());
+                Misc.freeObjListAndClear(partitionByRecord.getFunctions());
core/src/main/java/io/questdb/griffin/engine/functions/window/MinTimestampWindowFunctionFactory.java (1)

187-189: Readability nit: simplify zero-row window check.

-                else if (rowsLo == 0 && rowsLo == rowsHi) {
+                else if (rowsLo == 0 && rowsHi == 0) {
core/src/main/java/io/questdb/griffin/engine/functions/window/MinLongWindowFunctionFactory.java (1)

187-189: Readability nit: simplify zero-row window check.

-                else if (rowsLo == 0 && rowsLo == rowsHi) {
+                else if (rowsLo == 0 && rowsHi == 0) {
core/src/main/java/io/questdb/griffin/engine/functions/window/MaxTimestampWindowFunctionFactory.java (3)

462-463: Typo in comment: "undobuned" should be "unbounded"

-    // Handles max() over (partition by x order by ts range between [undobuned | y] preceding and [z preceding | current row])
+    // Handles max() over (partition by x order by ts range between [unbounded | y] preceding and [z preceding | current row])

441-444: Consider using putLong for consistency

The code uses putTimestamp to store a long value, while other similar implementations use putLong. While both work for TIMESTAMP columns, using putLong would be more consistent with the rest of the codebase.

-                    if (comparator.compare(l, value.getTimestamp(0))) {
-                        value.putTimestamp(0, l);
+                    if (comparator.compare(l, value.getLong(0))) {
+                        value.putLong(0, l);
                     }
                 } else {
-                    value.putTimestamp(0, l);
+                    value.putLong(0, l);

1676-1676: Inconsistency in column type definition

The MAX_COLUMN_TYPES uses ColumnType.TIMESTAMP to store the max value, but the variable name and usage suggest it's storing a long value. This could be confusing for maintainers.

-        MAX_COLUMN_TYPES.add(ColumnType.TIMESTAMP); // max value
+        MAX_COLUMN_TYPES.add(ColumnType.LONG); // max value
core/src/main/java/io/questdb/griffin/engine/functions/window/MaxLongWindowFunctionFactory.java (4)

462-463: Typo in comment: "undobuned" should be "unbounded"

-    // Handles max() over (partition by x order by ts range between [undobuned | y] preceding and [z preceding | current row])
+    // Handles max() over (partition by x order by ts range between [unbounded | y] preceding and [z preceding | current row])

441-444: Inconsistent method usage: putTimestamp vs putLong

In MaxOverPartitionFunction, you're using putTimestamp to store a long value when the function returns long. This is inconsistent with other implementations.

                 if (!value.isNew()) {
                     if (comparator.compare(l, value.getLong(0))) {
-                        value.putTimestamp(0, l);
+                        value.putLong(0, l);
                     }
                 } else {
-                    value.putTimestamp(0, l);
+                    value.putLong(0, l);

518-519: Inconsistent use of putTimestamp with long value

Similar inconsistency in MaxMinOverUnboundedPartitionRowsFrameFunction.

                     long max = value.getLong(0);
                     if (comparator.compare(l, max)) {
-                        value.putTimestamp(0, l);
+                        value.putLong(0, l);
                         max = l;

882-882: Potential refactoring opportunity for map value field access

In MaxMinOverPartitionRowsFrameFunction, when frameLoBounded is false, you're using value.getTimestamp(2) and value.putTimestamp(2, max) to store a long value at index 2. Consider using putLong/getLong for consistency.

                 } else {
-                    value.putTimestamp(2, this.maxMin);
+                    value.putLong(2, this.maxMin);
                 }
                     long max = value.getLong(2);
                     if (hiValue != Numbers.LONG_NULL) {
                         if (max == Numbers.LONG_NULL || comparator.compare(hiValue, max)) {
                             max = hiValue;
-                            value.putTimestamp(2, max);
+                            value.putLong(2, max);
                         }

Also applies to: 914-914

core/src/main/java/io/questdb/griffin/engine/functions/window/LastValueLongWindowFunctionFactory.java (2)

2031-2031: Typo in plan output: "between and" should be "and"

-                sink.val(" preceding between and ");
+                sink.val(" preceding and ");

601-604: Inconsistent use of putTimestamp with long values

Similar to MaxLongWindowFunctionFactory, there's inconsistent use of putTimestamp to store long values throughout this file.

For example:

                 if (d != Numbers.LONG_NULL) {
                     MapValue value = key.createValue();
-                    value.putTimestamp(0, d);
+                    value.putLong(0, d);
                 }
-            long val = value != null ? value.getTimestamp(0) : Numbers.LONG_NULL;
+            long val = value != null ? value.getLong(0) : Numbers.LONG_NULL;

Also applies to: 614-614

core/src/main/java/io/questdb/griffin/engine/functions/window/LastValueTimestampWindowFunctionFactory.java (2)

954-956: Nit: simplify ring-buffer copy length expression.

((firstIdx + size) % size) equals firstIdx; use firstIdx for clarity.

-                        Vect.memcpy(newAddress + firstPieceSize, oldAddress, ((firstIdx + size) % size) * RECORD_SIZE);
+                        Vect.memcpy(newAddress + firstPieceSize, oldAddress, firstIdx * RECORD_SIZE);

And similarly in LastValueOverRangeFrameFunction.

Also applies to: 1871-1873


686-693: Consider overflow-safe timestamp diffs.

Using Math.abs(timestamp - ts) can overflow for extreme values. Prefer a safe abs-diff helper if available (e.g., Numbers.absDiff or equivalent).

Also applies to: 716-723, 1839-1847

core/src/main/java/io/questdb/griffin/engine/functions/window/FirstValueLongWindowFunctionFactory.java (2)

215-225: Fix column-metadata comments to match actual usage.

Map layout here is: 0=oldest idx, 1=start offset, 2=first-not-null idx, 3=count (unbounded). Comments currently list "count" twice.

-                    columnTypes.add(ColumnType.LONG);// position of current oldest element
-                    columnTypes.add(ColumnType.LONG);// start offset of native array
-                    columnTypes.add(ColumnType.LONG);// count of values in buffer
-                    columnTypes.add(ColumnType.LONG);// count of values in buffer
+                    columnTypes.add(ColumnType.LONG);// position of current oldest element
+                    columnTypes.add(ColumnType.LONG);// start offset of native array
+                    columnTypes.add(ColumnType.LONG);// first not-null index
+                    columnTypes.add(ColumnType.LONG);// count (only when frameLoUnBounded)

167-168: Nit: update copy/paste comments ("moving average") to "first_value".

These comments are misleading.

-                    // moving average over range between timestamp - rowsLo and timestamp + rowsHi (inclusive)
+                    // first_value over range between timestamp - rowsLo and timestamp + rowsHi (inclusive)
-                    // moving average over range between timestamp - rowsLo and timestamp + rowsHi (inclusive)
+                    // first_value over range between timestamp - rowsLo and timestamp + rowsHi (inclusive)
-                    // moving average over preceding N rows
+                    // first_value over preceding N rows

Also applies to: 372-373, 437-439

core/src/main/java/io/questdb/griffin/engine/functions/window/FirstValueTimestampWindowFunctionFactory.java (3)

981-986: Prefer putTimestamp for TIMESTAMP map slots (clarity).

Semantics are identical, but putTimestamp improves readability and type intent.

-                    mapValue.putLong(0, d);
+                    mapValue.putTimestamp(0, d);
-                value.putLong(0, firstValue);
+                value.putTimestamp(0, firstValue);
-                mapValue.putLong(0, d);
+                mapValue.putTimestamp(0, d);

Also applies to: 1116-1121, 1893-1896


167-168: Nit: fix "moving average" copy/paste comments.

Use "first_value" to avoid confusion.

-                    // moving average over range between timestamp - rowsLo and timestamp + rowsHi (inclusive)
+                    // first_value over range between timestamp - rowsLo and timestamp + rowsHi (inclusive)
-                    // moving average over range between timestamp - rowsLo and timestamp + rowsHi (inclusive)
+                    // first_value over range between timestamp - rowsLo and timestamp + rowsHi (inclusive)
-                    // moving average over preceding N rows
+                    // first_value over preceding N rows

Also applies to: 372-373, 437-438


805-813: Consider overflow-safe timestamp diffs.

Same note as in last_value: Math.abs(timestamp - ts) can overflow; prefer a safe abs-diff helper.

Also applies to: 829-836, 1596-1610

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 26b9855 and 9c23898.

📒 Files selected for processing (10)
  • core/src/main/java/io/questdb/griffin/engine/functions/window/AbstractWindowFunctionFactory.java (2 hunks)
  • core/src/main/java/io/questdb/griffin/engine/functions/window/FirstValueLongWindowFunctionFactory.java (1 hunks)
  • core/src/main/java/io/questdb/griffin/engine/functions/window/FirstValueTimestampWindowFunctionFactory.java (1 hunks)
  • core/src/main/java/io/questdb/griffin/engine/functions/window/LastValueLongWindowFunctionFactory.java (1 hunks)
  • core/src/main/java/io/questdb/griffin/engine/functions/window/LastValueTimestampWindowFunctionFactory.java (1 hunks)
  • core/src/main/java/io/questdb/griffin/engine/functions/window/MaxLongWindowFunctionFactory.java (1 hunks)
  • core/src/main/java/io/questdb/griffin/engine/functions/window/MaxTimestampWindowFunctionFactory.java (1 hunks)
  • core/src/main/java/io/questdb/griffin/engine/functions/window/MinLongWindowFunctionFactory.java (1 hunks)
  • core/src/main/java/io/questdb/griffin/engine/functions/window/MinTimestampWindowFunctionFactory.java (1 hunks)
  • core/src/main/java/io/questdb/griffin/engine/table/AsyncGroupByNotKeyedRecordCursorFactory.java (1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (28)
  • GitHub Check: New pull request (SelfHosted Griffin And Fuzz tests on linux-arm64)
  • GitHub Check: New pull request (SelfHosted Other tests on linux-x64-zfs)
  • GitHub Check: New pull request (SelfHosted Griffin And Fuzz tests on linux-x64-zfs)
  • GitHub Check: New pull request (SelfHosted Other tests on linux-arm64)
  • GitHub Check: New pull request (SelfHosted Other tests Start X64Zfs Agent)
  • GitHub Check: New pull request (Hosted Running tests on windows-other)
  • GitHub Check: New pull request (Hosted Running tests on windows-pgwire)
  • GitHub Check: New pull request (Hosted Running tests on windows-cairo)
  • GitHub Check: New pull request (Hosted Running tests on windows-fuzz2)
  • GitHub Check: New pull request (Hosted Running tests on windows-fuzz1)
  • GitHub Check: New pull request (Hosted Running tests on windows-griffin)
  • GitHub Check: New pull request (Hosted Running tests on mac-other)
  • GitHub Check: New pull request (Hosted Running tests with cover on linux-other)
  • GitHub Check: New pull request (Hosted Running tests on mac-pgwire)
  • GitHub Check: New pull request (Hosted Running tests with cover on linux-pgwire)
  • GitHub Check: New pull request (Hosted Running tests on mac-cairo-fuzz)
  • GitHub Check: New pull request (Hosted Running tests with cover on linux-cairo)
  • GitHub Check: New pull request (Hosted Running tests on mac-cairo)
  • GitHub Check: New pull request (Hosted Running tests with cover on linux-fuzz)
  • GitHub Check: New pull request (Hosted Running tests on mac-griffin)
  • GitHub Check: New pull request (SelfHosted Griffin And Fuzz tests Start X64Zfs Agent)
  • GitHub Check: New pull request (SelfHosted Griffin And Fuzz tests Start ARM Agent)
  • GitHub Check: New pull request (Hosted Running tests with cover on linux-griffin)
  • GitHub Check: New pull request (SelfHosted Other tests Start ARM Agent)
  • GitHub Check: New pull request (Rust Test and Lint on linux-jdk17)
  • GitHub Check: New pull request (Check Changes Check changes)
  • GitHub Check: build
  • GitHub Check: build
🔇 Additional comments (6)
core/src/main/java/io/questdb/griffin/engine/table/AsyncGroupByNotKeyedRecordCursorFactory.java (1)

293-294: Approve — change correct; audit remaining Misc.freeObjList(...) usages and convert where lists may be reused/iterated.

Misc.freeObjListAndClear(groupByFunctions) fixes the null-hole NPE in AsyncGroupByNotKeyedRecordCursorFactory. A repo scan shows many remaining Misc.freeObjList(...) calls (e.g., FunctionParser.java, AsyncGroupByAtom.java, GroupByNotKeyedRecordCursorFactory.java, VirtualRecordCursorFactory.java, SqlCodeGenerator.java). Review each occurrence and replace with Misc.freeObjListAndClear(...) where the list can be iterated or reused after close.

core/src/main/java/io/questdb/griffin/engine/functions/window/AbstractWindowFunctionFactory.java (2)

178-194: Unbox zeroValue in LongNullFunction — LGTM.

Using primitive long removes boxing and aligns with Unsafe writes. No behavior change risk here.


196-213: Introduce TimestampNullFunction — LGTM.

Consistent with other null fillers; getTimestamp/pass1 mirror LongNullFunction semantics for timestamp columns.

core/src/main/java/io/questdb/griffin/engine/functions/window/LastValueTimestampWindowFunctionFactory.java (1)

72-97: Factory wiring and null-handling look correct.

Signature, validation, IGNORE/RESPECT NULLS split, and TimestampNullFunction fallback for invalid frames are consistent with other factories.

core/src/main/java/io/questdb/griffin/engine/functions/window/FirstValueLongWindowFunctionFactory.java (1)

64-96: API and validation flow look good.

Signature "first_value(L)", window validation, and null-frame fallback via LongNullFunction are correct.

core/src/main/java/io/questdb/griffin/engine/functions/window/FirstValueTimestampWindowFunctionFactory.java (1)

69-96: Factory setup looks consistent.

Signature "first_value(N)", validation, and TimestampNullFunction fallback are aligned with the design.

@bluestreak01 bluestreak01 changed the title feat(sql): first_value(timestamp) windows function feat(sql): add timestamp and long windows functions Sep 11, 2025
@bluestreak01 bluestreak01 added the SQL Issues or changes relating to SQL execution label Sep 11, 2025
@bluestreak01
Copy link
Copy Markdown
Member Author

@coderabbitai generate docstrings

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Sep 11, 2025

✅ Actions performed

Docstrings generation started.

coderabbitai bot added a commit that referenced this pull request Sep 11, 2025
Docstrings generation was requested by @bluestreak01.

* #6130 (comment)

The following files were modified:

* `core/src/main/java/io/questdb/griffin/engine/functions/window/AbstractWindowFunctionFactory.java`
* `core/src/main/java/io/questdb/griffin/engine/functions/window/FirstValueLongWindowFunctionFactory.java`
* `core/src/main/java/io/questdb/griffin/engine/functions/window/FirstValueTimestampWindowFunctionFactory.java`
* `core/src/main/java/io/questdb/griffin/engine/functions/window/LastValueLongWindowFunctionFactory.java`
* `core/src/main/java/io/questdb/griffin/engine/functions/window/LastValueTimestampWindowFunctionFactory.java`
* `core/src/main/java/io/questdb/griffin/engine/functions/window/MaxLongWindowFunctionFactory.java`
* `core/src/main/java/io/questdb/griffin/engine/functions/window/MaxTimestampWindowFunctionFactory.java`
* `core/src/main/java/io/questdb/griffin/engine/functions/window/MinLongWindowFunctionFactory.java`
* `core/src/main/java/io/questdb/griffin/engine/functions/window/MinTimestampWindowFunctionFactory.java`
* `core/src/main/java/io/questdb/griffin/engine/table/AsyncGroupByNotKeyedRecordCursorFactory.java`
@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Sep 11, 2025

Note

Generated docstrings for this pull request at #6131

@bluestreak01
Copy link
Copy Markdown
Member Author

@CodeRabbit review

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Sep 12, 2025

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Sep 12, 2025

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
2 out of 3 committers have signed the CLA.

✅ kafka1991
✅ bluestreak01
❌ coderabbitai[bot]
You have signed the CLA already but the status is still pending? Let us recheck it.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (5)
core/src/main/java/io/questdb/griffin/engine/functions/window/MinTimestampWindowFunctionFactory.java (5)

313-341: Constructor not guarded; mem/dequeMem leak if it throws.

Wrap the constructor along with allocations. This repeats a prior comment.

-                    MemoryARW mem = null;
-                    MemoryARW dequeMem = null;
-                    try {
-                        mem = Vm.getCARWInstance(
-                                configuration.getSqlWindowStorePageSize(),
-                                configuration.getSqlWindowStoreMaxPages(),
-                                MemoryTag.NATIVE_CIRCULAR_BUFFER
-                        );
-                        if (rowsLo != Long.MIN_VALUE) {
-                            dequeMem = Vm.getCARWInstance(
-                                    configuration.getSqlWindowStorePageSize(),
-                                    configuration.getSqlWindowStoreMaxPages(),
-                                    MemoryTag.NATIVE_CIRCULAR_BUFFER
-                            );
-                        }
-                    } catch (Throwable e) {
-                        Misc.free(mem);
-                        Misc.free(dequeMem);
-                        throw e;
-                    }
-                    return new MaxTimestampWindowFunctionFactory.MaxMinOverRowsFrameFunction(
-                            args.get(0),
-                            rowsLo,
-                            rowsHi,
-                            mem,
-                            dequeMem,
-                            LESS_THAN,
-                            NAME
-                    );
+                    MemoryARW mem = null;
+                    MemoryARW dequeMem = null;
+                    try {
+                        mem = Vm.getCARWInstance(
+                                configuration.getSqlWindowStorePageSize(),
+                                configuration.getSqlWindowStoreMaxPages(),
+                                MemoryTag.NATIVE_CIRCULAR_BUFFER
+                        );
+                        if (rowsLo != Long.MIN_VALUE) {
+                            dequeMem = Vm.getCARWInstance(
+                                    configuration.getSqlWindowStorePageSize(),
+                                    configuration.getSqlWindowStoreMaxPages(),
+                                    MemoryTag.NATIVE_CIRCULAR_BUFFER
+                            );
+                        }
+                        return new MaxTimestampWindowFunctionFactory.MaxMinOverRowsFrameFunction(
+                                args.get(0),
+                                rowsLo,
+                                rowsHi,
+                                mem,
+                                dequeMem,
+                                LESS_THAN,
+                                NAME
+                        );
+                    } catch (Throwable th) {
+                        Misc.free(mem);
+                        Misc.free(dequeMem);
+                        throw th;
+                    }

191-205: Same leak pattern: protect map allocation/constructor.

Wrap to ensure map is freed on constructor failure.

-                    Map map = MapFactory.createUnorderedMap(
-                            configuration,
-                            partitionByKeyTypes,
-                            MaxTimestampWindowFunctionFactory.MAX_COLUMN_TYPES
-                    );
-
-                    return new MaxTimestampWindowFunctionFactory.MaxMinOverPartitionFunction(
-                            map,
-                            partitionByRecord,
-                            partitionBySink,
-                            args.get(0),
-                            LESS_THAN,
-                            NAME
-                    );
+                    Map map = null;
+                    try {
+                        map = MapFactory.createUnorderedMap(
+                                configuration,
+                                partitionByKeyTypes,
+                                MaxTimestampWindowFunctionFactory.MAX_COLUMN_TYPES
+                        );
+                        return new MaxTimestampWindowFunctionFactory.MaxMinOverPartitionFunction(
+                                map,
+                                partitionByRecord,
+                                partitionBySink,
+                                args.get(0),
+                                LESS_THAN,
+                                NAME
+                        );
+                    } catch (Throwable th) {
+                        Misc.free(map);
+                        throw th;
+                    }

86-100: Free map on constructor failure (possible native leak).

Wrap map allocation + constructor in try/catch.

-                    Map map = MapFactory.createUnorderedMap(
-                            configuration,
-                            partitionByKeyTypes,
-                            MaxTimestampWindowFunctionFactory.MAX_COLUMN_TYPES
-                    );
-
-                    return new MaxTimestampWindowFunctionFactory.MaxMinOverPartitionFunction(
-                            map,
-                            partitionByRecord,
-                            partitionBySink,
-                            args.get(0),
-                            LESS_THAN,
-                            NAME
-                    );
+                    Map map = null;
+                    try {
+                        map = MapFactory.createUnorderedMap(
+                                configuration,
+                                partitionByKeyTypes,
+                                MaxTimestampWindowFunctionFactory.MAX_COLUMN_TYPES
+                        );
+                        return new MaxTimestampWindowFunctionFactory.MaxMinOverPartitionFunction(
+                                map,
+                                partitionByRecord,
+                                partitionBySink,
+                                args.get(0),
+                                LESS_THAN,
+                                NAME
+                        );
+                    } catch (Throwable th) {
+                        Misc.free(map);
+                        throw th;
+                    }

102-116: Same leak pattern: free map if constructor throws.

Apply the same try/catch as above.

-                    Map map = MapFactory.createUnorderedMap(
-                            configuration,
-                            partitionByKeyTypes,
-                            MaxTimestampWindowFunctionFactory.MAX_COLUMN_TYPES
-                    );
-
-                    return new MaxTimestampWindowFunctionFactory.MaxMinOverUnboundedPartitionRowsFrameFunction(
-                            map,
-                            partitionByRecord,
-                            partitionBySink,
-                            args.get(0),
-                            LESS_THAN,
-                            NAME
-                    );
+                    Map map = null;
+                    try {
+                        map = MapFactory.createUnorderedMap(
+                                configuration,
+                                partitionByKeyTypes,
+                                MaxTimestampWindowFunctionFactory.MAX_COLUMN_TYPES
+                        );
+                        return new MaxTimestampWindowFunctionFactory.MaxMinOverUnboundedPartitionRowsFrameFunction(
+                                map,
+                                partitionByRecord,
+                                partitionBySink,
+                                args.get(0),
+                                LESS_THAN,
+                                NAME
+                        );
+                    } catch (Throwable th) {
+                        Misc.free(map);
+                        throw th;
+                    }

171-186: Same leak pattern: protect map allocation/constructor.

Wrap to avoid leaking native map if constructor throws.

-                    Map map = MapFactory.createUnorderedMap(
-                            configuration,
-                            partitionByKeyTypes,
-                            MaxTimestampWindowFunctionFactory.MAX_COLUMN_TYPES
-                    );
-
-                    return new MaxTimestampWindowFunctionFactory.MaxMinOverUnboundedPartitionRowsFrameFunction(
-                            map,
-                            partitionByRecord,
-                            partitionBySink,
-                            args.get(0),
-                            LESS_THAN,
-                            NAME
-                    );
+                    Map map = null;
+                    try {
+                        map = MapFactory.createUnorderedMap(
+                                configuration,
+                                partitionByKeyTypes,
+                                MaxTimestampWindowFunctionFactory.MAX_COLUMN_TYPES
+                        );
+                        return new MaxTimestampWindowFunctionFactory.MaxMinOverUnboundedPartitionRowsFrameFunction(
+                                map,
+                                partitionByRecord,
+                                partitionBySink,
+                                args.get(0),
+                                LESS_THAN,
+                                NAME
+                        );
+                    } catch (Throwable th) {
+                        Misc.free(map);
+                        throw th;
+                    }
🧹 Nitpick comments (2)
core/src/main/java/io/questdb/griffin/engine/functions/window/MinTimestampWindowFunctionFactory.java (2)

306-308: Nit: readability on zero-row frame check.

Use rowsHi == 0 for clarity.

-                else if (rowsLo == 0 && rowsLo == rowsHi) {
+                else if (rowsLo == 0 && rowsHi == 0) {

Also applies to: 186-189


46-50: Optional: add minimal Javadoc.

Short class/function Javadoc improves discoverability in IDEs.

 public class MinTimestampWindowFunctionFactory extends AbstractWindowFunctionFactory {
+    /**
+     * MIN over TIMESTAMP window function factory.
+     * Mirrors MAX implementation with LESS_THAN comparator.
+     */
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9c23898 and b776f0b.

📒 Files selected for processing (5)
  • core/src/main/java/io/questdb/griffin/engine/functions/window/FirstValueLongWindowFunctionFactory.java (1 hunks)
  • core/src/main/java/io/questdb/griffin/engine/functions/window/LastValueLongWindowFunctionFactory.java (1 hunks)
  • core/src/main/java/io/questdb/griffin/engine/functions/window/LastValueTimestampWindowFunctionFactory.java (1 hunks)
  • core/src/main/java/io/questdb/griffin/engine/functions/window/MinLongWindowFunctionFactory.java (1 hunks)
  • core/src/main/java/io/questdb/griffin/engine/functions/window/MinTimestampWindowFunctionFactory.java (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (4)
  • core/src/main/java/io/questdb/griffin/engine/functions/window/MinLongWindowFunctionFactory.java
  • core/src/main/java/io/questdb/griffin/engine/functions/window/FirstValueLongWindowFunctionFactory.java
  • core/src/main/java/io/questdb/griffin/engine/functions/window/LastValueLongWindowFunctionFactory.java
  • core/src/main/java/io/questdb/griffin/engine/functions/window/LastValueTimestampWindowFunctionFactory.java
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (28)
  • GitHub Check: New pull request (SelfHosted Other tests on linux-arm64)
  • GitHub Check: New pull request (SelfHosted Griffin And Fuzz tests on linux-arm64)
  • GitHub Check: New pull request (SelfHosted Other tests on linux-x64-zfs)
  • GitHub Check: New pull request (SelfHosted Griffin And Fuzz tests on linux-x64-zfs)
  • GitHub Check: New pull request (Hosted Running tests on windows-other)
  • GitHub Check: New pull request (Hosted Running tests on windows-pgwire)
  • GitHub Check: New pull request (Hosted Running tests on windows-cairo)
  • GitHub Check: New pull request (Hosted Running tests on windows-fuzz2)
  • GitHub Check: New pull request (Hosted Running tests on windows-fuzz1)
  • GitHub Check: New pull request (Hosted Running tests on windows-griffin)
  • GitHub Check: New pull request (Hosted Running tests on mac-other)
  • GitHub Check: New pull request (Hosted Running tests on mac-pgwire)
  • GitHub Check: New pull request (Hosted Running tests on mac-cairo-fuzz)
  • GitHub Check: New pull request (SelfHosted Other tests Start X64Zfs Agent)
  • GitHub Check: New pull request (Hosted Running tests on mac-cairo)
  • GitHub Check: New pull request (SelfHosted Griffin And Fuzz tests Start X64Zfs Agent)
  • GitHub Check: New pull request (Rust Test and Lint on linux-jdk17)
  • GitHub Check: New pull request (Hosted Running tests on mac-griffin)
  • GitHub Check: New pull request (SelfHosted Griffin And Fuzz tests Start ARM Agent)
  • GitHub Check: New pull request (SelfHosted Other tests Start ARM Agent)
  • GitHub Check: New pull request (Hosted Running tests with cover on linux-other)
  • GitHub Check: New pull request (Hosted Running tests with cover on linux-pgwire)
  • GitHub Check: New pull request (Hosted Running tests with cover on linux-cairo)
  • GitHub Check: New pull request (Hosted Running tests with cover on linux-fuzz)
  • GitHub Check: New pull request (Hosted Running tests with cover on linux-griffin)
  • GitHub Check: New pull request (Check Changes Check changes)
  • GitHub Check: build
  • GitHub Check: build
🔇 Additional comments (7)
core/src/main/java/io/questdb/griffin/engine/functions/window/MinTimestampWindowFunctionFactory.java (7)

72-80: LGTM: Null window result handling.

Returning TimestampNullFunction for rowsHi < rowsLo is correct and consistent.


124-168: LGTM: Range-over-partition with proper cleanup.

Allocation is guarded; all resources freed on failure.


187-189: LGTM: Current-row fast path.

Direct current-row function is appropriate.


208-250: LGTM: Rows-over-partition with proper cleanup.

Allocations are guarded and freed in catch.


254-261: LGTM: No-order default/range and unbounded-rows handling.

Behavior aligns with MAX variant semantics.


262-300: RANGE with non-default frame requires ORDER BY; verify behavior when not ordered.

This branch proceeds even if not ordered, using timestampIndex. Ensure timestampIndex is valid; otherwise either:

  • enforce ORDER BY for non-default RANGE frames, or
  • route to whole-result set only for default frames.

Consider:

-                else {
-                    if (windowContext.isOrdered() && !windowContext.isOrderedByDesignatedTimestamp()) {
+                else {
+                    if (!windowContext.isOrdered()) {
+                        throw SqlException.$(windowContext.getOrderByPos(), "RANGE frame requires ORDER BY");
+                    }
+                    if (!windowContext.isOrderedByDesignatedTimestamp()) {
                         throw SqlException.$(windowContext.getOrderByPos(), "RANGE is supported only for queries ordered by designated timestamp");
                     }

46-55: Keep "min(N)" — matches existing timestamp factories.
MinTimestampWindowFunctionFactory uses SIGNATURE = NAME + "(N)" and MaxTimestampWindowFunctionFactory uses the same; do not change to "(T)".

@ideoma ideoma changed the title feat(sql): add timestamp and long windows functions feat(sql): add timestamp and long window functions Sep 12, 2025
@kafka1991 kafka1991 self-requested a review September 15, 2025 01:49
kafka1991
kafka1991 previously approved these changes Sep 15, 2025
Copy link
Copy Markdown
Collaborator

@kafka1991 kafka1991 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

kafka1991
kafka1991 previously approved these changes Sep 15, 2025
Copy link
Copy Markdown
Collaborator

@kafka1991 kafka1991 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@glasstiger
Copy link
Copy Markdown
Contributor

[PR Coverage check]

😍 pass : 192 / 309 (62.14%)

file detail

path covered line new line coverage
🔵 io/questdb/griffin/engine/functions/window/AbstractWindowFunctionFactory.java 0 6 00.00%
🔵 io/questdb/griffin/engine/functions/window/LastValueDoubleWindowFunctionFactory.java 1 6 16.67%
🔵 io/questdb/griffin/engine/functions/window/AvgDoubleWindowFunctionFactory.java 2 6 33.33%
🔵 io/questdb/griffin/engine/functions/window/MinTimestampWindowFunctionFactory.java 68 136 50.00%
🔵 io/questdb/griffin/engine/functions/window/CountFunctionFactoryHelper.java 4 8 50.00%
🔵 io/questdb/griffin/engine/functions/window/FirstValueDoubleWindowFunctionFactory.java 4 8 50.00%
🔵 io/questdb/griffin/engine/functions/window/MaxDoubleWindowFunctionFactory.java 4 6 66.67%
🔵 io/questdb/griffin/engine/functions/window/MinLongWindowFunctionFactory.java 101 125 80.80%
🔵 io/questdb/griffin/engine/functions/window/CountConstWindowFunctionFactory.java 2 2 100.00%
🔵 io/questdb/griffin/engine/table/AsyncGroupByNotKeyedRecordCursorFactory.java 1 1 100.00%
🔵 io/questdb/griffin/engine/functions/window/MinDoubleWindowFunctionFactory.java 2 2 100.00%
🔵 io/questdb/griffin/engine/functions/window/SumDoubleWindowFunctionFactory.java 2 2 100.00%
🔵 io/questdb/griffin/SqlCodeGenerator.java 1 1 100.00%

@bluestreak01 bluestreak01 merged commit fd461c2 into master Sep 15, 2025
34 of 35 checks passed
@bluestreak01 bluestreak01 deleted the vi_first_timestamp branch September 15, 2025 12:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

SQL Issues or changes relating to SQL execution

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NPE when preparing async group by

4 participants