Skip to content

feat(sql): enable SQL projection columns to reference preceding columns on the same projection#5778

Merged
bluestreak01 merged 101 commits intomasterfrom
vi_projection
Jul 4, 2025
Merged

feat(sql): enable SQL projection columns to reference preceding columns on the same projection#5778
bluestreak01 merged 101 commits intomasterfrom
vi_projection

Conversation

@bluestreak01
Copy link
Copy Markdown
Member

@bluestreak01 bluestreak01 commented Jun 23, 2025

Projection expressions are allowed to reference preceding aliases from the same projection. In case of projection alias clashing with a column name from the base SQL, the latter is preferred. E.g. this is the expected output:

> CREATE TABLE test (x integer);
> INSERT INTO test VALUES (1), (2), (3);
> select x + 10 x, x - 5 from test;
┌───────┬─────────┐
│   x   │ (x - 5) │
│ int32 │  int32  │
├───────┼─────────┤
│    11-4 │
│    12-3 │
│    13-2 │
└───────┴─────────┘
>

Memoizable functions

There are two new methods on Function interface:

Function.shouldMemoize()
Function.memoizeI()

The immediate beneficiary is the family of rnd_ functions, which execution is now stabilised per data row.

todo

Parser/Optimiser support for

  • select f(a) i, i + 1 from ... - vanilla
  • select f(a) (i), f(i) (y), sum(y) from ... group-by key and value
  • select f(a) i, f(i) y, sum(y) from tab sample by 1d - sample by
  • select f(a) i, lag(i) over from ... - window functions

Generator support for

  • vanilla
  • window

@puzpuzpuz puzpuzpuz self-requested a review July 4, 2025 15:39
puzpuzpuz
puzpuzpuz previously approved these changes Jul 4, 2025
puzpuzpuz
puzpuzpuz previously approved these changes Jul 4, 2025
@glasstiger
Copy link
Copy Markdown
Contributor

[PR Coverage check]

😍 pass : 1136 / 1326 (85.67%)

file detail

path covered line new line coverage
🔵 io/questdb/griffin/engine/functions/rnd/RndLogFunctionFactory.java 0 1 00.00%
🔵 io/questdb/griffin/model/QueryModel.java 3 12 25.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndSymbolListFunctionFactory.java 2 5 40.00%
🔵 io/questdb/griffin/model/QueryColumn.java 1 2 50.00%
🔵 io/questdb/griffin/engine/functions/memoization/FloatFunctionMemoizer.java 23 39 58.97%
🔵 io/questdb/cairo/sql/Function.java 2 3 66.67%
🔵 io/questdb/griffin/engine/functions/TernaryFunction.java 8 12 66.67%
🔵 io/questdb/griffin/engine/functions/BinaryFunction.java 7 10 70.00%
🔵 io/questdb/griffin/engine/functions/memoization/IPv4FunctionMemoizer.java 29 39 74.36%
🔵 io/questdb/griffin/engine/functions/memoization/DateFunctionMemoizer.java 29 39 74.36%
🔵 io/questdb/griffin/engine/functions/memoization/TimestampFunctionMemoizer.java 29 39 74.36%
🔵 io/questdb/griffin/engine/functions/memoization/CharFunctionMemoizer.java 29 39 74.36%
🔵 io/questdb/griffin/engine/functions/QuaternaryFunction.java 15 20 75.00%
🔵 io/questdb/griffin/engine/functions/UnaryFunction.java 6 8 75.00%
🔵 io/questdb/griffin/engine/functions/memoization/IntFunctionMemoizer.java 30 39 76.92%
🔵 io/questdb/griffin/engine/functions/memoization/LongFunctionMemoizer.java 30 39 76.92%
🔵 io/questdb/griffin/engine/functions/memoization/ByteFunctionMemoizer.java 30 39 76.92%
🔵 io/questdb/griffin/engine/functions/memoization/ShortFunctionMemoizer.java 30 39 76.92%
🔵 io/questdb/griffin/engine/functions/memoization/DoubleFunctionMemoizer.java 30 39 76.92%
🔵 io/questdb/griffin/engine/functions/memoization/BooleanFunctionMemoizer.java 31 39 79.49%
🔵 io/questdb/griffin/engine/functions/memoization/Long256FunctionMemoizer.java 50 63 79.37%
🔵 io/questdb/griffin/engine/functions/memoization/UuidFunctionMemoizer.java 38 48 79.17%
🔵 io/questdb/griffin/engine/groupby/GroupByUtils.java 60 73 82.19%
🔵 io/questdb/griffin/PriorityMetadata.java 19 21 90.48%
🔵 io/questdb/cairo/sql/VirtualFunctionRecord.java 46 51 90.20%
🔵 io/questdb/griffin/engine/table/VirtualRecordCursorFactory.java 29 30 96.67%
🔵 io/questdb/griffin/SqlOptimiser.java 261 269 97.03%
🔵 io/questdb/griffin/engine/functions/rnd/RndVarcharFunctionFactory.java 1 1 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndIPv4CCFunctionFactory.java 2 2 100.00%
🔵 io/questdb/griffin/engine/functions/date/TimestampSequenceFunctionFactory.java 1 1 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndLongCCFunctionFactory.java 2 2 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndCharFunctionFactory.java 2 2 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndSymbolFunctionFactory.java 1 1 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndBinCCCFunctionFactory.java 1 1 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndDoubleFunctionFactory.java 3 3 100.00%
🔵 io/questdb/griffin/engine/join/OuterJoinRecord.java 3 3 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndLong256NFunctionFactory.java 1 1 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndVarcharListFunctionFactory.java 1 1 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndStringListFunctionFactory.java 1 1 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndUuidCCFunctionFactory.java 2 2 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndIntervalFunctionFactory.java 1 1 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndByteCCFunctionFactory.java 1 1 100.00%
🔵 io/questdb/griffin/SqlParser.java 3 3 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndGeoHashFunctionFactory.java 5 5 100.00%
🔵 io/questdb/griffin/engine/groupby/AbstractSampleByFillRecordCursor.java 2 2 100.00%
🔵 io/questdb/griffin/FunctionParser.java 15 15 100.00%
🔵 io/questdb/griffin/engine/window/WindowRecordCursorFactory.java 1 1 100.00%
🔵 io/questdb/griffin/engine/table/VirtualFunctionRecordCursor.java 55 55 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndLong256FunctionFactory.java 2 2 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndIntFunctionFactory.java 3 3 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndIPv4FunctionFactory.java 2 2 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndDoubleCCFunctionFactory.java 2 2 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndLongFunctionFactory.java 3 3 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndDateCCCFunctionFactory.java 1 1 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndShortFunctionFactory.java 3 3 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndStrFunctionFactory.java 1 1 100.00%
🔵 io/questdb/cairo/sql/NullRecord.java 1 1 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndBinFunctionFactory.java 1 1 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndShortCCFunctionFactory.java 2 2 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndFloatFunctionFactory.java 2 2 100.00%
🔵 io/questdb/griffin/engine/groupby/SampleByFillRecord.java 1 1 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndBooleanFunctionFactory.java 2 2 100.00%
🔵 io/questdb/griffin/engine/groupby/SampleByFillValueRecordCursor.java 1 1 100.00%
🔵 io/questdb/griffin/SqlCodeGenerator.java 101 101 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndTimestampFunctionFactory.java 2 2 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndIntCCFunctionFactory.java 2 2 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndUuidFunctionFactory.java 2 2 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndStrFunction.java 1 1 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndStrRndListFunctionFactory.java 1 1 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndDateFunctionFactory.java 2 2 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndByteFunctionFactory.java 2 2 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndDoubleArrayFunctionFactory.java 2 2 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndFloatCFunctionFactory.java 2 2 100.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndVarcharFunction.java 1 1 100.00%
🔵 io/questdb/griffin/engine/functions/MultiArgFunction.java 19 19 100.00%
🔵 io/questdb/griffin/engine/functions/json/JsonExtractFunction.java 1 1 100.00%

@bluestreak01 bluestreak01 merged commit 2fba4f0 into master Jul 4, 2025
34 checks passed
@bluestreak01 bluestreak01 deleted the vi_projection branch July 4, 2025 19:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Enhancement Enhance existing functionality SQL Issues or changes relating to SQL execution

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants