QuestDB 9.3.4

QuestDB 9.3.4 delivers dynamic windows in WINDOW JOIN, Parquet row group pruning with bloom filters, new array functions, and significant performance improvements across ORDER BY, joins, and Parquet I/O.

For any questions or feedback, please join us on Slack or on Discourse.

See also our prettier release notes page.

Breaking changes 💥

• Parquet Varchar encoding default changed: Parquet export now uses RLE Dictionary encoding for Varchar columns instead of Delta Length Byte Array. Parquet files written by 9.3.4 use the new encoding by default. If downstream tools or pipelines depend on the previous encoding, use the new per-column encoding config to override.
  • by @RaphDal in #6809
• Constant expression folding: The SQL engine now folds constant expressions at compile time, which aligned compile-time behaviour of NaN/Infinity values with their runtime behaviour. Infinity and -Infinity in constant float/double expressions are now collapsed to NULL at compile time, consistent with QuestDB's existing NULL convention. CASE/SWITCH expressions can no longer branch on Infinity or -Infinity as distinct values.
  • by @RaphDal in #6828

Highlights

Dynamic window support in WINDOW JOIN

WINDOW JOIN now supports dynamic window ranges computed from column values or expressions:

SELECT t.*,
       avg(p.mid) AS avg_mid
FROM trades t
WINDOW JOIN prices p
  ON p.sym = t.sym
  RANGE BETWEEN t.lookback PRECEDING
        AND CURRENT ROW
  INCLUDE PREVAILING;

Parquet row group pruning with bloom filters

Queries over Parquet files now leverage min/max statistics and bloom filters to skip entire row groups that cannot match the query predicate, dramatically reducing I/O for selective queries.

Array functions

New DOUBLE[] functions: array_sort(), array_reverse(), array_elem_min(), array_elem_max(), array_elem_avg(), and array_elem_sum().

Other

• arg_min() and arg_max() now support CHAR arguments.
• Per-column Parquet encoding/compression configuration.
• minTimestamp and maxTimestamp columns added to sys.telemetry_wal.

Performance

• ASOF and WINDOW JOIN: Faster execution for large right-hand-side tables.
• ORDER BY: Pre-computed sort keys for fixed-width types; dedicated fast path for SYMBOL columns.
• HORIZON JOIN: Better parallelization across various data distributions.
• Vectorized GROUP BY: More non-keyed queries now use SIMD computation.
• Parquet decoding: Faster row group decompression and column materialization.
• Parquet writing: Reduced write amplification and improved write speed.
• WAL writer: New cairo.wal.writer.madvise.mode config option for tuning memory access patterns.

Bug fixes

• Fixed data corruption in DECIMAL128 and DECIMAL256 columns.
• Fixed crash on LATEST BY ALL queries over large tables.
• Fixed crash when Parquet partition statistics are missing.
• Fixed crash in SAMPLE BY FILL with array column aggregates.
• Fixed read_parquet() crash on SYMBOL columns from native Parquet files.
• Fixed WINDOW JOIN INCLUDE PREVAILING dropping the prevailing row when the window had matches.
• Fixed WINDOW JOIN dropping the prevailing row on cross-partition boundaries.
• Fixed AssertionError triggered by certain JOIN queries.
• Fixed read_parquet() on Parquet files with stale QuestDB metadata.
• Fixed support for quoted column names in ALTER COLUMN.
• Fixed resource leaks and NPEs in the SQL engine.
• Disabled materialized view parallel SQL on low-core machines.

Changelist

• fix(core): fix crash on LATEST BY ALL queries over large tables by @ideoma in #6832
• feat(core): add minTimestamp and maxTimestamp columns to sys.telemetry_wal by @ideoma in #6779
• fix(parquet): fix crash when parquet partition statistics are missing by @kafka1991 in #6827
• feat(sql): add array_sort and array_reverse functions for double arrays by @javier in #6820
• perf(wal): add cairo.wal.writer.madvise.mode configuration option by @jerrinot in #6841
• perf(sql): speed up ASOF and WINDOW JOINs for large right-hand-side tables by @puzpuzpuz in #6822
• fix(sql): fix SAMPLE BY FILL crash with array column aggregates by @javier in #6811
• perf(parquet): improve decoding performance for parquet by @RaphDal in #6759
• feat(sql): array_elem_{min,max,avg,sum} functions by @amunra in #6829
• perf(sql): breaking change 💥 - Improve query execution time by folding constant expressions by @RaphDal in #6828
• fix(sql): fix AssertionError that could be triggered by JOIN SQL by @DHRUV6029 in #6824
• fix(sql): support quoted column names in ALTER COLUMN by @nwoolmer in #6842
• feat(core): parquet row group pruning with min/max statistics and bloom filters by @kafka1991 in #6739
• feat(sql): use vectorized computation in more non-keyed GROUP BY queries by @puzpuzpuz in #6805
• fix(sql): fix WINDOW JOIN INCLUDE PREVAILING dropping prevailing row when window has matches by @RaphDal in #6868
• fix(core): fix read_parquet() crash on SYMBOL columns from native parquet files by @ideoma in #6865
• perf(parquet): breaking change 💥 - improve strings decoding performance for parquet files by @RaphDal in #6809
• perf(sql): improve ORDER BY performance for fixed-width column types by @kafka1991 in #6862
• feat(sql): arg_min and arg_max CHAR functions by @nwoolmer in #6730
• perf(sql): speed up ORDER BY on SYMBOL columns by @kafka1991 in #6870
• fix(sql): WINDOW JOIN dropping prevailing row on cross-partition by @RaphDal in #6871
• perf(sql): optimize parallel HORIZON JOIN for various data distributions by @puzpuzpuz in #6867
• fix(sql): fix bugs, resource leaks and NPE in SQL engine by @bluestreak01 in #6874
• feat(sql): add dynamic window support in WINDOW JOIN by @puzpuzpuz in #6859
• fix(core): fix data corruption in DECIMAL128 and DECIMAL256 columns by @jerrinot in #6873
• feat(core): control parquet writing file size growth, reduce write amplification, improve write speed by @ideoma in #6819
• feat(sql): add per-column parquet encoding/compression config by @RaphDal in #6843
• fix(core): disable materialized view parallel SQL on low-core machines by @nwoolmer in #6891
• fix(sql): fix read_parquet on Parquet files with stale QDB metadata by @nwoolmer in #6885

Full Changelog: 9.3.3...9.3.4

QuestDB 9.3.3

QuestDB 9.3.3 is a feature-rich release introducing HORIZON JOIN for markout analysis, a new twap() aggregate, SQL-standard WINDOW definitions, JIT compilation on ARM64, and file-based secrets for Kubernetes deployments. It also brings significant performance improvements across Parquet I/O, parallel GROUP BY, UNION queries, and ORDER BY on computed expressions.

For any questions or feedback, please join us on Slack or on Discourse.

See also our prettier release notes page.

Highlights

HORIZON JOIN for markout analysis

HORIZON JOIN is a new join type designed for markout analysis — a common financial analytics pattern where you measure how prices or metrics evolve at specific time offsets relative to events like trades or orders.

For each row in the left-hand table and each offset in the horizon, the join computes left_timestamp + offset and performs an ASOF match against the right-hand table. Results are implicitly grouped by the horizon offset and any specified keys, with aggregate functions applied across all matched rows.

Here's an example measuring post-trade price impact at 1-second intervals up to 60 seconds:

SELECT h.offset / 1_000_000 AS horizon_sec, t.sym, avg(m.mid) AS avg_mid
FROM trades AS t
HORIZON JOIN mid_prices AS m ON (t.sym = m.sym)
RANGE FROM 1s TO 60s STEP 1s AS h
ORDER BY t.sym, horizon_sec

You can also use LIST for non-uniform horizons and negative offsets to look at pre-event behavior:

SELECT h.offset / 1_000_000 AS horizon_sec, t.sym,
       avg(m.mid - t.price) AS avg_markout
FROM trades AS t
HORIZON JOIN mid_prices AS m ON (t.sym = m.sym)
LIST (-5s, -1s, 0, 1s, 5s, 30s, 1m) AS h
ORDER BY t.sym, horizon_sec

The horizon pseudo-table exposes h.offset (raw microsecond value) and h.timestamp (the computed left_timestamp + offset), which can be used in expressions and grouping.

twap() time-weighted average price

The new twap(price, timestamp) aggregate computes the time-weighted average price using step-function integration: each price is held constant until the next observation, and the TWAP is the area under the step function divided by the total time span. It supports parallel GROUP BY and SAMPLE BY with FILL modes.

SELECT symbol, twap(price, timestamp) AS twap_price
FROM trades
WHERE timestamp IN today()
SAMPLE BY 1h;

WINDOW definition clause

QuestDB now supports the SQL-standard WINDOW clause for defining reusable window specifications. Instead of repeating the same PARTITION BY and ORDER BY in multiple window function calls, define them once and reference by name:

SELECT symbol, side, price,
       sum(price) OVER ws, avg(price) OVER ws,
       sum(amount) OVER ws, sum(price) OVER wt
FROM trades
WINDOW wt AS (ORDER BY timestamp),
       ws AS (PARTITION BY symbol, side ORDER BY timestamp);

Window inheritance is also supported, where a named window references another as its base:

SELECT avg(price) OVER w2
FROM trades
WINDOW w1 AS (PARTITION BY symbol ORDER BY ts),
       w2 AS (w1 ROWS BETWEEN 1 PRECEDING AND CURRENT ROW)

Chained inheritance and standard merge rules for PARTITION BY, ORDER BY, and frame clauses are all supported.

JIT compilation on ARM64

QuestDB's JIT filter compiler now runs natively on ARM64 (aarch64) systems, including Apple Silicon and AWS Graviton. Previously, JIT-compiled filters were only available on x86. Benchmarks on an Apple M5 show filter evaluations running up to 2x faster with JIT enabled for common WHERE clause patterns, with OR-predicate filters seeing up to 5x improvement.

File-based secrets for Kubernetes

Sensitive configuration like database passwords can now be loaded from files using the _FILE suffix convention, enabling integration with Kubernetes Secrets, Docker Secrets, and HashiCorp Vault:

export QDB_PG_PASSWORD_FILE=/run/secrets/pg_password

File-based secrets are automatically trimmed, visible in SHOW PARAMETERS with value_source = 'file', and reloaded when contents change via SELECT reload_config().

array_build() function

The new array_build(nDims, size, filler1, ...) function creates DOUBLE[] or DOUBLE[][] arrays with controlled shape and fill values. The size parameter accepts a scalar integer or a DOUBLE[] (using its cardinality), and each filler can be a scalar (repeated) or an array (copied element-by-element with NaN padding or truncation).

Performance

Parquet I/O

• Parquet partition reads with selective filters now use late materialization — filter columns are decoded first, and remaining columns are decoded only for matching rows. This gives up to a 2.4x speedup on OHLC-style aggregation queries with symbol filters.
• Parquet writing has been sped up.
• Parquet export of queries with computed expressions (e.g., CAST, arithmetic, string functions) no longer falls back to a temporary table. A new hybrid export mode passes raw columns zero-copy and materializes only computed columns into native buffers.
• Decimal type support (DECIMAL8 through DECIMAL256) has been added for Parquet read and write.

Query execution

• Parallel GROUP BY and Top K queries now use unordered page frame collection, eliminating head-of-line blocking. At concurrency 8, per-iteration thread spread drops by 53–75%, tail latency (p99) drops by 30–44%, and latency predictability (p99/p50 ratio) improves from 2.5–3.6x down to 1.4–1.8x. There is a 34% single-thread regression for keyed GROUP BY that does not appear under concurrent load.
• UNION, UNION ALL, EXCEPT, and INTERSECT queries now push designated-timestamp filters into each branch, enabling per-branch partition pruning.
• CASE WHEN expressions on symbol columns now resolve string constants to integer symbol keys at init time and compare by int at runtime.
• ORDER BY on computed expressions now pre-computes sort key values into off-heap buffers, reducing function evaluations from O(N log N) to O(N).
• Double-to-decimal conversion performance has been improved with bulk multiply optimization.

Security

• Fixed a server crash caused by a crafted HTTP request with an overflowing chunk size in chunked transfer encoding. The hex chunk-size parser now rejects values that would overflow a 64-bit integer.

Changelist

• fix(core): suppress error output in print-hello.sh during startup failure by @ideoma in #6738
• feat(core): support reading secrets from files for Kubernetes deployments by @bluestreak01 in #6719
• feat(core): optimize parquet partition read with late materialization, zero-copy page reading, and use raw array encoding by @kafka1991 in #6675
• fix(http): prevent parquet export corruption under concurrent connections by @jerrinot in #6723
• perf(parquet): speed up parquet writing by @ideoma in #6735
• feat(sql): allow underscore number separator in TICK syntax by @mtopolnik in #6749
• fix(core): correct greedy parsing for single N pattern in timestamp formats by @jerrinot in #6741
• fix(core): fix intermittent backup failures on Windows caused by file access problems by @RaphDal in #6748
• fix(core): clear changedKeys at start of config reload to prevent stale notifications by @bluestreak01 in #6756
• fix(sql): fix ADD COLUMN IF NOT EXISTS for DECIMAL, GEOHASH, and array types by @mtopolnik in #6753
• feat(core): add JIT support for aarch64 by @RaphDal in #6758
• feat(sql): window definition by @javier in #6746
• fix(sql): reject trailing content after valid query by @mtopolnik in #6751
• fix(core): fix assertion errors in UNION ALL with column count mismatch by @mtopolnik in #6744
• fix(core): add dedicated batch size config for parquet export by @ideoma in #6747
• fix(sql): fix table and name-lock leak on CREATE TABLE AS SELECT failure by @kafka1991 in #6763
• fix(ilp): fix assertion error in ILP decimal parser on malformed input by @bluestreak01 in #6772
• perf(sql): improve performance of UNION and friends by pushing down timestamp filter into subqueries by @mtopolnik in #6745
• fix(sql): fix to recognise view names in quotes by @glasstiger in #6771
• fix(sql): fix alias hiding original column name in CASE with window function by @mtopolnik in #6775
• fix(sql): compile tick expressions with date variables into IR for zero-parse evaluation when executing caches SQL statements by @bluestreak01 in #6780
• fix(core): fix checkpoint timeout caused by slow sync() system call by @glasstiger in #6785
• perf(sql): speed up parallel GROUP BY and Top K queries by @mtopolnik in #6754
• feat(sql): add twap() time-weighted average price aggregate function by @bluestreak01 in #6786
• feat(parquet): add decimal type support for parquet read/write by @RaphDal in #6725
• perf(sql): optimize CASE WHEN on symbol columns to co...

QuestDB 9.3.2

QuestDB 9.3.2 continues the trend of performance upgrades and bugfixes, with some additional new features. Importantly, we introduce the new TICK syntax, a compact DSL for expressing time intervals, alongside faster aggregations, improved applicability of interval scans, and fast parquet queries.

For any questions or feedback, please join us on Slack or on Discourse.

See also our prettier release notes page.

Highlights

New TICK syntax

The Temporal Interval Calendar Kit (TICK is a new DSL for expressing complex time ranges and intervals in a compact, easy-to-use format.

For example, let's say that you want to query one month of data from NYSE, only including trading days and hours. The data in the database is stored in UTC format, so you'd need to convert to different time zones, and build a very complex WHERE clause with ANDs and ORs. Or, alternatively, send lots of narrow queries and combine the results.

Instead, you can express this in a simple string:

-- NYSE trading hours on workdays for January
SELECT * FROM trades
WHERE ts IN '2024-01-[01..31]T09:30@America/New_York#workday;6h29m';

Reading this from left to right, we will the days in 2024-01 in the range of 01..31 (all of them), with the interval beginning at 09:30 in New York time, only considering working days, and the interval ending after 6h29m on each day (inclusive, so at 16:00 New York time).

We then compile this to an efficient interval scan, with the time filtering pushed down:

 intervals: [
    ("2024-01-01T14:30:00.000000Z","2024-01-01T20:59:59.999999Z"),     

    ("2024-01-02T14:30:00.000000Z","2024-01-02T20:59:59.999999Z"),

    ("2024-01-03T14:30:00.000000Z","2024-01-03T20:59:59.999999Z"),

    ...
]

This syntax makes it easier to construct complex intervals, and keep confidence that the execution plan will be optimal.

Please see the TICK docs for more information.

arg_min, arg_max, and geomean aggregates

Let's say you are monitoring trading data, and you track what the max trading price was over an hourly period. You can express that like this:

SELECT timestamp, symbol, max(price)
FROM trades
WHERE timestamp IN today()
AND symbol IN ('BTC-USDT', 'ETH-USDT')
SAMPLE BY 1h;

But now we have a problem - at what time or trade was that the max price?

Using arg_max, we can extract values from the row where the price was max:

SELECT timestamp, symbol, max(price), arg_max(timestamp, price)
FROM trades
WHERE timestamp IN today()
AND symbol IN ('BTC-USDT', 'ETH-USDT')
SAMPLE BY 1h;

Pretty handy!

geomean(D) calculates the geometric mean for a set of positive numbers; this is a useful average variant, which improves accuracy for data with large outliers, and commonly used for growth rate calculations.

For more details, please check out the aggregate function documentation.

EMA, VWEMA, percent_rank window functions

We've introduced new exponential moving average (EMA) window functions. This smooths out your data base on a smoothing and time period.

Additionally, we've added a volume-weighted variant (VWEMA), which is commonly used for trading data, allowing for smoothing of the function whilst still prioritising higher-volume trades.

Here's an example of the syntax:

-- Standard average
avg(value) OVER (window_definition)

-- Exponential Moving Average (EMA)
avg(value, kind, param) OVER (window_definition)

-- Volume-Weighted Exponential Moving Average (VWEMA)
avg(value, kind, param, volume) OVER (window_definition)

percent_rank returns the relative rank of a row within a group of values. This is calculated based on the number of rows within the group, and its position.

For more information, please see the window functions documentation.

Web Console

We've added new improvements to the 'QuestDB AI Assistant' feature, improving the speed and responsiveness of the chat. We'll be following up soon with an expansion of supported model providers.

Performance

Time-intrinsics upgrades

• QuestDB will now optimise time-range queries over tables where the range uses a constant dateadd offset.
  • e.g. WHERE dateadd('m', 15, timestamp) = '2022-03-08T18:30:00Z'
• QuestDB will optimise time filters that include OR clauses:
  • e.g. WHERE timestamp IN '2018-01-01' OR timestamp IN '2018-01-02' OR timestamp IN '2018-01-03'
• We've improved how time-range pushdown is handled within nested queries:
  • e.g. SELECT * FROM (SELECT dateadd('h', -1, timestamp) as ts, price FROM trades) WHERE ts in '2022'

Parquet and File-handling

• We have significantly sped-up querying parquet queries by optimising how row-groups are decoded, giving up to a 6x performance boost versus the last release.
• We've optimised how file mappings are handled, removing sporadic query latency increases over extremely concurrent query workloads.

Changelist

• feat(sql): volume-weighted exponential moving average (VWEMA) window function by @bluestreak01 in #6651
• feat(sql): add arg_min() and arg_max() aggregate functions by @bluestreak01 in #6652
• feat(sql): implement EMA window function via avg() by @bluestreak01 in #6650
• fix(ilp): prevent writes going to the wrong place after a table rename by @nwoolmer in #6654
• feat(sql): add bool_and() and bool_or() aggregate functions by @bluestreak01 in #6658
• feat(sql): add bit_and(), bit_or(), and bit_xor() aggregate functions by @bluestreak01 in #6660
• feat(sql): add geomean() aggregate function by @bluestreak01 in #6656
• fix(sql): prevent SIGSEGV in window join by awaiting workers before freeing cache by @bluestreak01 in #6662
• feat(sql): recognize dateadd predicates in time intrinsic filters by @puzpuzpuz in #6666
• fix(sql): potential segfault on vectorized GROUP BY query timeout by @puzpuzpuz in #6667
• feat(sql): support nested window functions with deduplication optimization by @bluestreak01 in #6643
• feat(sql): add within_box, within_radius, and geo_within_radius_latlon functions by @bluestreak01 in #6664
• feat(sql): recognize OR of timestamp IN predicates as interval intrinsics by @bluestreak01 in #6673
• fix(sql): handle unbalanced quotes in SQL line comments by @bluestreak01 in #6684
• perf(core): avoid contention across concurrent queries on expensive file operation by @mtopolnik in #6688
• perf(sql): apply scan optimization for first/last/min/max(timestamp) as function argument by @kafka1991 in #6690
• fix(sql): error on group by with many aggregate functions by @puzpuzpuz in #6689
• feat(sql): add length_bytes() function for varchars by @puzpuzpuz in #6685
• fix(sql): fix reverse-order argument parsing in multi-arg window function by @mtopolnik in #6697
• fix(sql): proper error message when timestamp used along another join key in ASOF/LT join by @mtopolnik in #6698
• fix(sql): fix an issue where the DECLAREd symbols weren't honored when parsing function arguments by @mtopolnik in #6700
• fix(sql): fix parquet read failure on chained window join by @kafka1991 in #6676
• fix(sql): resolve 'Invalid column' error in WINDOW JOIN with aliased columns by @bluestreak01 in #6701
• feat(sql): TICK - Temporal Interval Calendar Kit for interval literals by @bluestreak01 in #6674
• fix(pgwire): prepared batch with ...

QuestDB 9.3.1

QuestDB 9.3.1 follows the major 9.3.0 release, focusing on stability, correctness, and performance refinements based on early feedback and production usage.

This release delivers targeted fixes across joins, views, and checkpointing, alongside continued performance improvements on hot SQL execution paths.

Improvements

Arithmetic expressions in window functions

Window functions now support arithmetic expressions directly, allowing analytical queries to compute derived values inline without requiring subqueries or post-processing:

SELECT
  symbol,
  price,
  price - lag(price) OVER (PARTITION BY symbol ORDER BY ts) AS delta
FROM trades

This simplifies common patterns such as calculating deltas, ratios, and scaled values within window definitions.

Extended tables() metadata

The tables() system view now exposes two additional columns:

• table_min_timestamp
• table_max_timestamp

These columns provide quick visibility into the temporal bounds of each table, useful for diagnostics, retention checks, and operational tooling.

ksum() window function

The ksum() function now works as a window function, using the Kahan summation algorithm for improved floating-point precision. This complements the existing ksum() aggregate function by enabling its use in window contexts:

-- Cumulative sum with reduced floating-point error
SELECT ksum(price) OVER (ORDER BY ts ROWS UNBOUNDED PRECEDING) FROM trades;

-- Sliding window
SELECT ksum(price) OVER (ORDER BY ts ROWS BETWEEN 3 PRECEDING AND CURRENT ROW) FROM trades;

-- Partitioned
SELECT ksum(price) OVER (PARTITION BY symbol) FROM trades;

All standard window frame types are supported: ROWS, RANGE, partitioned, unbounded, and sliding windows.

Performance

Streaming Parquet export

Parquet exports via the HTTP /exp endpoint now stream directly from page frames, eliminating intermediate temporary tables. This reduces memory overhead and improves export throughput for large result sets.

Reduced garbage on parallel query hot paths

Parallel query execution has been optimized to reduce garbage generation on hot paths. This lowers GC pressure and improves throughput and tail latency under sustained analytical workloads.

Avoid repeated expression execution

Queries where columns reference other columns now avoid redundant expression evaluation. This improves performance for wide projections and queries with multiple derived columns.

Bug fixes

SQL and query execution

• Fixed an error when using WINDOW JOIN (introduced in 9.3.0) with ORDER BY ts DESC.
• Fixed a crash in ASOF JOIN queries when the ON clause mixes SYMBOL and non-symbol columns.
• Fixed transient "file does not exist" errors that could surface during query execution.

Views and materialized views

• Fixed an issue where views (introduced in 9.3.0) could become suspended after being altered.
• Fixed a rare bug that could write invalid data into a materialized view under specific conditions.

Core, storage, and checkpoints

• Fixed checkpoint restore logic by removing phantom table directories left on disk.
• Fixed a rare issue where process memory was not fully released on Linux when jemalloc is enabled.

Thanks to everyone who reported issues, shared production feedback, and contributed fixes and improvements. Your input continues to shape QuestDB's reliability and performance.

For questions or feedback, please join us on Slack or Discourse, and check the full changelog on GitHub for detailed PR information.

What's Changed

• feat(sql): implement ksum() window function with Kahan summation by @bluestreak01 in #6642
• perf(core): streaming parquet export by @kafka1991 in #6300
• feat(sql): implement arithmetic with window functions by @bluestreak01 in #6626
• fix(sql): WINDOW JOIN with ORDER BY ts DESC returns error by @puzpuzpuz in #6624
• fix(core): fix checkpoint restore by removing phantom tables directories from disk by @ideoma in #6614
• perf(sql): reduce garbage generated on parallel query hot path by @puzpuzpuz in #6597
• fix(sql): fix view metadata race conditions on replica by @bluestreak01 in #6627
• fix(core): fix for view becomes suspended after it is altered by @glasstiger in #6623
• fix(core): fix rare bug that can potentially write invalid data in mat view by @ideoma in #6628
• perf(sql): reduce repeated expression execution when a column is referenced by other columns by @kafka1991 in #6093
• fix(core): fix transient file does not exist error in queries by @ideoma in #6629
• fix(sql): fix ASOF JOIN crash when ON clause has symbol and other columns by @jerrinot in #6634
• fix(core): process memory may not be released on Linux when jemalloc is enabled by @puzpuzpuz in #6619
• feat(sql): add table_min_ and table_max_timestamp columns to tables() view by @bluestreak01 in #6630

Full Changelog: 9.3.0...9.3.1

QuestDB 9.3.0

QuestDB 9.3.0 is now available, bringing a new wave of query expressiveness and usability improvements across the engine and the Web Console. This release introduces window joins for precise time-based analytics, database views for cleaner query composition, AI-assisted workflows in the web console, and the new PIVOT keyword for effortless wide-schema aggregations.

For any questions or feedback, please join us on Slack or on Discourse.

See also our prettier release notes page.

New Features

Views

Views are virtual tables defined by a SELECT statement. They do not persist data on disk. Instead, the underlying query is executed as a subquery each time the view is referenced.

Complex calculations and logic can be captured in a view, and shared across database users.

CREATE VIEW latest_trades AS
SELECT
  symbol,
  side,
  last(price) AS price,
  last(timestamp) AS ts
FROM trades
GROUP BY symbol, side;

Views can be queried just like tables, while always reflecting the latest data from their source tables.

Parameterisation

Views can be parameterised:

CREATE VIEW filtered_trades AS (
  DECLARE 
      OVERRIDABLE @min_price := 100
  SELECT ts, symbol, price 
  FROM trades 
  WHERE price >= @min_price
);

This view contains the @min_price parameter, with a default value. Additionally, its tagged as OVERRIDABLE. This means that the user reading from the View can alter this parameter:

DECLARE 
    @min_price := 500 
SELECT * FROM filtered_trades

Within a View, DECLARE variables are constant unless explicitly declared to be OVERRIDABLE. This allows you to create Views with pre-set filters, limiting which rows the end user has access to:

CREATE VIEW last_24h_trades AS (
   DECLARE 
       @min_ts := dateadd('d', -1, now())
   SELECT * FROM trades 
   WHERE timestamp >= @min_ts
)

The end user cannot widen this lower time bounds parameter.

Please see the docs for more information.

PIVOT

This release introduces the new PIVOT keyword.

PIVOT is a specialised form of GROUP BY that transforms rows into columns, helping you move from a narrow schema to a wide one in a single query.

A traditional aggregation might look like this (try on demo!):

SELECT side, symbol, last(price) AS price
FROM trades_latest_1d
WHERE symbol IN ('BTC-USD', 'ETH-USD')
ORDER BY side, symbol;

Resulting in a narrow result set:

With PIVOT, the same aggregate can be computed and output with a column generated for each pivot key combination. In this case, the pivoting key is symbol, and the aggregate value is last(price).

trades_latest_1d
PIVOT (
  last(price)
  FOR symbol IN ('BTC-USD', 'ETH-USD')
  GROUP BY side
) ORDER BY side;

This makes it much easier to work with dashboarding and data analysis tools, where column-per-symbol is a natural analysis and charting format.

Dynamic PIVOT keys

PIVOT can generate columns based on keys generated by a query. For example:

trades_latest_1d
PIVOT (
  last(price)
  FOR symbol IN (SELECT DISTINCT symbol FROM trades_latest_1d)
  GROUP BY side
) ORDER BY side;

Please see the docs for more information.

Window joins (beta)

QuestDB now supports WINDOW JOIN, a new join syntax designed specifically for time-based analytics.

WINDOW JOIN allows each row from a primary table to be joined with a time window of rows from another table, with aggregations computed over the matching rows. This makes short-horizon analytics—such as correlating trades with nearby market prices—both expressive and efficient.

SELECT
  t.*,
  avg(p.bid) AS avg_bid,
  avg(p.ask) AS avg_ask
FROM trades t
WINDOW JOIN prices p
  ON p.sym = t.sym
  RANGE BETWEEN 1 second PRECEDING
        AND 1 second FOLLOWING
  INCLUDE PREVAILING;

In this example, each row from trades is joined with all rows from prices that share the same symbol and fall within the [-1s, +1s] interval (inclusive). Aggregations are then calculated over the joined rows.

This syntax avoids complex subqueries and makes time-windowed joins explicit, readable, and performant, with SIMD acceleration employed for the most common aggregate functions.

Please see the docs for more information.

AI-assisted workflows in the Web Console (beta)

The QuestDB Web Console now includes LLM-powered assistance, available on an opt-in basis using your own API key.

Once enabled, the console can:

• Auto-describe table schemas and column semantics
• Annotate sample rows with plain-language context
• Walk through query execution plans, highlighting joins, filters, and index usage

This makes it easier to understand both what a query does and how it executes—directly where you write SQL.

Performance

• JIT-compiled filters have undergone an optimisation pass, and will now execute up to 2x faster.
• Queries with GROUP BY, ORDER BY and LIMIT can now be executed in parallel, with execution times up to 10x faster.
• SELECT min(timestamp), max(timestamp) FROM table will now execute in O(1) time (instantly), when timestamp is the designated timestamp.
• Querying external parquet files with read_parquet('data.parquet') will now use projection pushdown, signficantly reducing disk and memory usage, and speeding up execution times by as much as 100x.
• Parquet exporting has been sped up by reducing the number of intermediate commits applied during the copy process.

Changelist

• fix(sql): segfault when using touch() on a table with a new empty column by @nwoolmer in #6561
• feat(sql): window join by @puzpuzpuz in #6292
• fix(pgwire): add support for SHOW default_transaction_read_only by @nwoolmer in #6562
• fix(core): fix incorrect designated timestamp column displayed by tables() after column type changes by @glasstiger in #6570
• fix(sql): support copies of very wide tables and result sets by @nwoolmer in #6525
• fix(sql): fix false timestamp ordering error with nested SAMPLE BY and ORDER BY by @bluestreak01 in #6572
• perf(sql): avoid expensive row counting in EXPLAIN query with LIMIT by @mtopolnik in #6540
• feat(sql): align glob() function with DuckDB glob syntax by @kafka1991 in #6552
• perf(sql): speed up JIT-compiled filters by reordering predicates and short-circuiting them by @puzpuzpuz in #6568
• feat(core): release disk space faster on table drop by @ideoma in #6555
• perf(sql): speed up row counting in filter queries by @puzpuzpuz in #6580
• feat(pgwire): support VARCHAR[] bind variables for symbol/varchar/str IN expressions by @kafka1991 in #6574
• feat(sql): add column projection pushdown for read_parquet() by @kafka1991 in #6551
• perf(sql): parallel ORDER BY long_column LIMIT N for high-cardinality GROUP BY by @puzpuzpuz in #6582
• fix(sql): fix EXPLAIN UPDATE resulting in error by @bluestreak01 in #6588
• fix(sql): handle quoted column names with dots in JOIN metadata by @jerrinot in #6590
• feat(sql): add rowCount, txn and timestamp columns to tables() by @bluestreak01 in #6581
• perf(sql): speed up min/max aggregates on designated timestamp by @puzpuzpuz in #6593
• fix(sql): inefficient commit batching during parquet exports by @nwoolmer in #6596
• fix(conf): auto-detect native libs in jlink runtime by @jerrinot in #6493
• fix(core): prevent rounding overflow from being ignored during decimal divisions by @RaphDal in #6598
• feat(sql): support PIVOT keyword for rotating rows to columns by @nwoolmer in #5313
• fix(pgwire): allow large varchar column regardless of send buffer size by @jerrinot in #6603
• chore(conf): claude.md to steer agents by @jerrinot in #6607
• feat(core): views by @glasstiger in #5720
• fix(sql): collect dependent columns during top-down projection propagation by @kafka1991 in https://github.com/questdb...

QuestDB 9.2.3 is a patch release bringing the usual swathe of fixes, performance enhancements, and a few new features. You'll see higher QPS, faster GROUP BY queries, and better performance for high-contention materialized views.

There will be one more release before Christmas, bringing the new WINDOW JOIN syntax. Stay tuned!

For any questions or feedback, please join us on Slack or on Discourse.

See also our prettier release notes page.

Highlights

• Materialized Views
  • Faster updates under high contention: views which update frequently (multiple times per second) will now use significantly less resources and have reduced refresh latency.
  • Faster historical refreshes: views with very small SAMPLE BY units will now refresh much more quickly, especially for historical or full refreshes.
• TTL
  • Additional safety checks: TTL will now use min(maxTimestamp, wallClockTime) as the predicate for dropping old partitions, rather than just maxTimestamp. This avoids accidental partition drops when erroneous timestamps are written to the table.
• SQL
  • Faster (de)allocations: Memory deallocations have been sped up significantly, with jemalloc now enabled by default on Linux builds. This particularly helps for reducing query tail latencies.
  • Faster query compilation: Query parsing performance has been improved by around ~4x. This reduces latency for lightweight and fast-to-execute queries, and maximises concurrent QPS when prepared statements are not used.
  • Faster single-key GROUP BY: GROUP BY queries that group by a single key will now execute ~2-3x faster.
    • e.g. SELECT symbol, count() as count FROM trades GROUP BY symbol ORDER BY count DESC LIMIT 10;
  • Faster non-keyed averages: Keyless GROUP BY queries will now execute ~1.3x faster.
    • e.g. SELECT AVG(price) FROM trades;
  • Decimal fills: SAMPLE BY with FILL now properly supports the new DECIMAL type.

Changelist

• fix(core): fix missing partition dir on table reader open when writing identical data with dedup by @ideoma in #6479
• fix(sql): crash when using parallel ORDER BY with LIMIT and JIT filter by @puzpuzpuz in #6482
• fix(ui): remove line ending declarations from web console assembly descriptor by @emrberk in #6490
• feat(core): hints for handling histogram out of bounds error for approx_percentile()/approx_median() by @peter007-cmd in #6446
• perf(sql): use unordered maps only in single-column case by @puzpuzpuz in #6481
• fix(sql): fix parquet export bug when SQL contains now() function by @kafka1991 in #6499
• fix(sql): fix empty result set from limit queries by @bluestreak01 in #6504
• fix(sql): fix potential crash when ORDER BY symbol column added via ALTER TABLE ADD COLUMN by @kafka1991 in #6505
• fix(sql): improve refresh step estimation for sparse data with small sample by interval by @kafka1991 in #6517
• chore(utils): update async-profiler to 4.2.1 by @jerrinot in #6473
• fix(core): data corruption after range replace on column top partition by @puzpuzpuz in #6519
• fix(http): correct HTTP content length for JSON responses by @kafka1991 in #6520
• perf(core): optimize mat view writing by skipping transactions that are fully replaced by future commits or full refresh (2) by @ideoma in #6495
• fix(core): fix bad connection state after TLS session initialization failure by @jerrinot in #6529
• fix(sql): fix potential unexpected exception when parquet file schema changed in read_parquet() by @kafka1991 in #6513
• feat(core): use wall clock for TTL to prevent accidental data loss by @bluestreak01 in #6531
• fix(sql): support for decimal types in group by fill and selected cursor by @RaphDal in #6532
• fix(sql): not to ignore outer limit if a subquery has a filter and another limit by @glasstiger in #6533
• perf(sql): reduce query latency from memory deallocation overhead on Linux x86_64 by @jerrinot in #6477
• fix(sql): fix symbol column nullValue flag when adding column to non-empty table by @kafka1991 in #6507
• perf(sql): improve latency of parsing SQL expressions by @bluestreak01 in #6538
• perf(sql): speed up avg() functions in non-keyed GROUP BY queries by @puzpuzpuz in #6542
• fix(core): UPDATE to work with bind variables for TIMESTAMP columns by @glasstiger in #6510
• fix(sql): allow aggregate functions inside CASE expressions by @bluestreak01 in #6550

New Contributors

• @peter007-cmd made their first contribution in #6446

Full Changelog: 9.2.2...9.2.3

QuestDB 9.2.2 is another stability release, primarily fixing some query snags. With this, we've packaged a performance improvement for vertically-scaled machines, and some new SQL functions.

For any questions or feedback, please join us on Slack or on Discourse.

See also our prettier release notes page.

Breaking changes 💥

• We've removed the legacy txn scoreboard (v1).
  • Using the new scoreboard (v2) has been the default for some time now.
  • If you have been manually overriding this config option (cairo.txn.scoreboard.format=1), it will no longer take effect, but the instance will boot and function as normal.
• MANUAL PERIOD views will now only be refreshed by manual trigger, and not also automatically when the time period ends.
  • This was always the intended behaviour; it is unexpected that a MANUAL view should refresh automatically.

SQL

New weighted statistics functions

• weighted_avg(DD) - computes the weighted average based on a 'value' column and a 'weights' column.
  • A straightforward weighted average calculation.
• weighted_stddev_rel(DD) - computes the weighted standard deviation based a 'value' column and a 'reliability weights' column.
  • Reliability weights are based on how trusted or reliable the particular value should be treated.
• weighted_stddev_freq(DD) - computes the weighted standard deviation based on a 'value' column and a 'frequency weights' column.
  • Frequency weights are based on the frequency of a particular sample occurring in the dataset.

Changelist

• fix(sql): fix bug where queries using IN clauses with INT/LONG columns containing -1 return incorrect results. by @kafka1991 in #6439
• fix(sql): fix internal error when concurrent threads try to drop a non-wal table by @jerrinot in #6462
• fix(pgwire): fix protocol corruption when sending arrays from server to client by @bluestreak01 in #6455
• fix(core): breaking change 💥 - remove tx scoreboard v1 by @jerrinot in #6449
• fix(sql): breaking change 💥 - inconsistent data in chained period materialized views by @puzpuzpuz in #6463
• fix(core): fix a bug where default ILP decimal wouldn't be null by @RaphDal in #6454
• fix(core): fix internal errors due to bad implicit casting by @jerrinot in #6443
• perf(sql): improve query scalability on large multicore machines by @puzpuzpuz in #6459
• fix(pgwire): batch SELECTs with bound parameters return correct results by @jerrinot in #6453
• feat(sql): weighted average and standard deviation by @mtopolnik in #6457

Full Changelog: 9.2.1...9.2.2

QuestDB 9.2.1 is a stability release, bringing a number of fixes, and some key performance enhancements, including markout horizon CROSS JOIN upgrades, and improvements to SQL query latencies across the board.

For any questions or feedback, please join us on Slack or on Discourse.

See also our prettier release notes page.

Highlights

Materialized Views

Materialized Views now have simplified syntax for building views with only-complete buckets, by specifying that the refresh period should match the sample by interval.

CREATE MATERIALIZED VIEW trades_daily_prices
REFRESH PERIOD (SAMPLE BY INTERVAL) AS
SELECT
  timestamp,
  symbol,
  avg(price) AS avg_price
FROM trades
SAMPLE BY 1d ALIGN TO CALENDAR TIME ZONE 'Europe/London';

In addition, PERIOD views now supported intervals in seconds.

CROSS JOIN

We've released a new optimisation for certain CROSS JOIN queries, specifically for markout analysis. This is a new query hint, which drastically speeds up building a market horizon table. Here is an example:

CREATE TABLE orders (
      ts TIMESTAMP,
      price DOUBLE
  ) timestamp(ts) PARTITION BY DAY;

INSERT INTO orders
  SELECT
      generate_series as timestamp,
      rnd_double() * 10.0 + 5.0
      FROM generate_series('2025-01-02', '2025-01-02T00:10', '200u');

WITH
  offsets AS (
    SELECT sec_offs, 10_000_000 * sec_offs usec_offs 
    FROM (SELECT x-61 AS sec_offs FROM long_sequence(121))
  )
  SELECT /*+ markout_horizon(orders offsets) */ sum(price) FROM (SELECT * FROM (
    SELECT price, ts + usec_offs AS timestamp
    FROM orders CROSS JOIN offsets
    ORDER BY ts + usec_offs
  ) TIMESTAMP(timestamp));

On an r7a.4xlarge instance:

This is part of a series of features focused around optimising post-trade analysis, and driven directly by user feedback, so keep it coming!

Performance

• Reduction in GC overhead for executing high-cardinality GROUP BY/SAMPLE BY queries.
• count_distinct() has been sped up, up to 2x based on standard benchmarks.
• Reduced memory usage for reads and writes over HTTP.
• General SQL query latency reduction by optimising munmap usage:
  • This is opt-in, and can be enabled by setting cairo.file.async.unmap.enabled=true

SQL

• New files(s) and glob(s) for scanning and filtering the server's filesystem.
• New glob(Ss) function, an alternative to LIKE and ~ that uses glob-style syntax.
• New first_not_null(array) and last_not_null(array) group by functions.
• New API endpoint for validating SQL queries: /api/v1/sql/validate.

Changelist

• fix(ilp): backwards incompatible way of writing null arrays by @puzpuzpuz in #6396
• perf(http): garbage-free HTTP header parser by @jerrinot in #6397
• fix(ilp): remove multi-url blacklisting code by @nwoolmer in #6393
• perf(sql): remove litter generated by ordered map rehashing by @puzpuzpuz in #6399
• fix(pgwire): support UUID bind variables in JIT filters by @jerrinot in #6413
• fix(sql): order-by ignored in sub-queries of aggregation by @RaphDal in #6414
• feat(sql): add first_not_null(array) by @mcadariu in #6344
• fix(sql): cannot compile query with aggregate function containing unary minus by @nwoolmer in #6404
• perf(sql): improve SQL latency by moving munmap() to a background job by @jerrinot in #6386
• fix(sql): query with limit x, y (x > 0 & y > 0) return correct size() by @kafka1991 in #6409
• feat(sql): add last_not_null(array) by @mcadariu in #6368
• fix(sql): fix add index issue for symbol columns in matviews by @kafka1991 in #6424
• feat(sql): implement files(s), glob(s) and glob(Ss) functions by @nwoolmer in #6391
• perf(sql): optimized Markout Horizon CROSS JOIN by @mtopolnik in #6283
• perf(sql): speed up keyed parallel GROUP BY in case of high cardinality count_distinct() by @puzpuzpuz in #6432
• fix(sql): fix sample by with only one fill option by @kafka1991 in #6437
• fix(core): fix rare suspended WAL table when ALTER and RENAME are executed concurrently by @ideoma in #6440
• fix(sql): fix a bug that made some ASOF JOIN queries fail with an internal error by @mtopolnik in #6433
• feat(http): REST API for SQL validation by @bluestreak01 in #6383
• fix(sql): fix a bug where a query hint in main SELECT would end up in CTEs as well by @mtopolnik in #6441
• fix(sql): limited subqueries within a union not skipping rows correctly by @nwoolmer in #6395
• fix(sql): export parquet support empty table/partition by @kafka1991 in #6420
• feat(sql): period sample by interval syntax for materialized views by @puzpuzpuz in #6428
• fix(http): negative questdb_json_queries_connections gauge by @nwoolmer in #6444

New Contributors

• @liuguoqingfz made their first contribution in #6242
• @Mittalkabir made their first contribution in #6260

Full Changelog: 9.2.0...9.2.1

✨ QuestDB 9.2: Native DECIMAL Type

QuestDB 9.2.0 brings the long-awaited DECIMAL type, a bespoke decimal type, declared using DECIMAL(precision, scale). Under the hood, the database automatically selects an optimal storage size, from DECIMAL8 (1 byte) up to DECIMAL256 (32 bytes). All of our usual arithmetical operations are supported out-of-the-box, making it easy to adopt and avoid precision loss.

With this new DECIMAL type, and our recently released support for nanosecond timestamps (TIMESTAMP_NS), QuestDB has never been a better fit for capital markets!

This release also brings upgrades to our already-fast ASOF JOIN queries, with a new Dense algorithm optimised for distant row matching. This marks our fourth ASOF JOIN algorithm, helping you to ensure your temporal join queries stay performant, regardless of your underlying data distribution.

For any questions or feedback, please join us on Slack or on Discourse.

See also our prettier release notes page.

Highlights

DECIMAL(precision, scale)

High-precision arithmetic

DECIMAL allows for arithmetic without precision loss, and only a slightly performance cost (2x for DECIMAL64 vs DOUBLE); aoperations which would silently lose precision with DOUBLE will instead warn the user, preserving data integrity.

-- With DOUBLE:
SELECT 0.1 + 0.2; -- returns: 0.30000000000000004 

-- With DECIMAL:
SELECT 0.1m + 0.2m;  -- returns: 0.3

Configurable scale

Configure precision and scale directly, without worrying about the underlying data type, with support for up to 76 digits!

CREATE TABLE transactions (
    id LONG,
    amount DECIMAL(14, 2),      -- Up to 999,999,999,999.99
    tax_rate DECIMAL(5, 4),      -- Up to 9.9999 (e.g., 0.0875 for 8.75%)
    quantity DECIMAL(10, 3),     -- Up to 9,999,999.999
    timestamp TIMESTAMP
) timestamp(timestamp);

API compatibility

DECIMAL is supported over PG Wire, ILP and HTTP APIs, with parquet support coming very soon.

Use cases

• Preserving integrity of data in ledgers, simplifying regulatory compliance
• Easy storage for cryptocurrency data, ordinarily requiring at least 18 digits of precision.
• No need to work around integer-scaling or integer signedness issues.

See the decimal concepts documentation for more information.

Dense ASOF JOIN

QuestDB 9.2.0 introduces a new 'dense' ASOF JOIN algorithm, optimised for datasets where the matching rows are likely to be far apart. This complements the existing ASOF JOIN algorithms to ensure that your queries can achieve the best performance regardless of your data distribution.

The choice of which algorithm to use is important, and explained in detail in the optimizer hints documentation.

Here is a summary of the algorithms:

• Fast; optimised for closely matching rows, using a binary-search and backwards scans.
• Memoized; the same as Fast, but more tolerant of occassional distant matches.
• Light; optimised when the RHS table has a highly selective filter, using forward-only scan on RHS table.
• Dense; optimised for distant matches, using a binary-search, bidirectional scans and memoized results.

We will continue to iterate on this feature in conjunction with the upcoming performance optimisations for markout analysis.

Changelist

• feat(sql): enable symbol capacity auto-scaling by default by @nwoolmer in #6352
• fix(core): prevent rare exception from async jobs when table is dropped by @bluestreak01 in #6361
• fix(core): prevent async jobs from getting SIGBUS error under extreme system load by @bluestreak01 in #6357
• fix(core): fix critical storage corruption on deduplicate write resulting to same data by @ideoma in #6359
• perf(sql): optimize ASOF JOIN for dense interleaving of left-hand and right-hand rows by @mtopolnik in #6362
• feat(core): support for Decimal by @RaphDal in #6068
• fix(core): fix occasional errors in result set size calculation by @glasstiger in #6351
• chore(core): correct Decimal type indices in ColumnTypeTag enum by @RaphDal in #6382
• feat(core): add a GC-free CharSequence -> int hash map for WalWriter symbols by @RaphDal in #6371
• fix(sql): always free parquet buffers when executing queries in parallel by @puzpuzpuz in #6372
• feat(core): move release to use Graal CE JDK by @bluestreak01 in #6390

Full Changelog: 9.1.1...9.2.0

QuestDB 9.1.1

QuestDB 9.1.1 focuses on data interoperability, nanosecond precision, and analytical performance.
This release introduces Parquet export, a faster ASOF JOIN for sparse symbol data, and several engine optimizations improving both query throughput and stability.

Highlights

Data Interoperability

• Parquet export: COPY TO now supports exporting query results directly to Parquet format for seamless integration with data-lake and analytics tools such as Spark, Snowflake, and Athena (#6008).
• Optimized Parquet import: read_parquet() recognizes timestamp-sorted data, significantly accelerating ingest for large, time-ordered datasets (#6079).

Precision

• Nanosecond precision ingestion: The Java ILP client now supports nanosecond timestamps, aligning with QuestDB’s highest-precision timekeeping (#6220).
• Synthetic data generation: New rnd_symbol_zipfn() function produces high-cardinality symbol distributions following a Zipf / Power Law — ideal for testing realistic workloads (#6227).

Performance

• ASOF JOIN optimization: Major speedup for joins on low-frequency symbols via a memory-efficient lookup strategy that reuses symbol search results (#6208, #6333).
• Query engine improvements:
  • Parallel count_distinct() now uses a lazy merge strategy, reducing memory usage (#6268).
  • Expanded fast path optimization for ordered and limited single-column queries (#6272).
  • Parallel SAMPLE BY execution for queries combining timestamps and aggregates (#6275).
  • Disabled column pre-touch in parallel filters, improving cache efficiency (#6280).

🧱 Parquet Export

QuestDB now supports exporting tables or queries to Parquet format using the COPY command — enabling direct data exchange with data lakes and analytics pipelines.

-- export an entire table to parquet, with same partitioning as original
COPY 'table_name' TO 'folder_name' WITH FORMAT PARQUET;

-- export a subquery to parquet with new partitioning
COPY (SELECT * FROM trades WHERE timestamp IN today()) TO 'folder_name' WITH FORMAT PARQUET PARTITION_BY MONTH;

Users can also parquet data ad-hoc using /exp?fmt=parquet (and omitting COPY).

Benefits

• Efficient columnar compression and encoding
• Open, widely supported format
• Perfect for interoperability with modern data ecosystems

🧮 New SQL Functions

• rnd_symbol_zipfn() – generates high-cardinality symbols following a Power Law distribution (#6227).
• last(array) – returns the last element from an array expression (#6291).
• Nanosecond support – added new nanos functions and improved CAST behavior for date types (#6247).

⚙️ SQL & Query Engine

• Recognize timestamp-sorted data in read_parquet() for faster imports (#6079).
• Fixed invalid column errors in GROUP BY on joined tables (#6275, #6332).
• Improved GROUP BY and SAMPLE BY to avoid duplicate group keys (#6275).
• Fixed incorrect results from SAMPLE BY with mixed timestamp / aggregate expressions (#6254).
• Corrected handling of negative offsets in SAMPLE BY (#6306).
• Fixed JIT compilation for WHERE sym1 = sym2 filters (#6321).
• Fixed overflow in materialized view refresh interval and step calculations (#6233, #6258).
• Interval constants can now be applied as time intrinsics (#6263).
• Prevented critical-level logs on group-by query cancellation (#6235).
• Fixed NullPointerException issues in window and optimizer code paths (#6264, #6265).

📥 Ingestion (ILP)

• Java ILP client now supports nanosecond precision timestamps (#6220).
• Fixed flaky LineHttpSenderLoggingTest stability issue (#6315).

🧩 Core & Operations

Resource & Reliability

• Added resource pool tracing to help detect memory leaks during development and testing (#6234).
• Fixed potential connection leaks on Windows (#6243).
• Fixed index error after column addition and truncation of non-partitioned tables (#6319).
• Fixed NullPointerException in ColumnPurgeJob (#6274).
• Improved reliability under high connection load (#6334).
• Added debug logging to WAL purge operations (#6336).

HTTP Server

• Relaxed cookie parser for legacy HTTP 1.0 and ANSI C timestamp formats while maintaining RFC 2616 compliance (#6290).

Web Console

• Updated to Web Console v1.1.4 (#6309).
• Fixed issue preventing configuration override on upgrade (#6240).

⚡ Performance & Stability

• Disabled column pre-touch in parallel filters for better cache utilization (#6280).
• Added new GroupByArraySink for optimized aggregation (#6237).
• Improved parallel group-by performance and stability (#6303).
• Optimized count_distinct() memory usage via lazy merge (#6268).
• Faster single-column ordered queries with expanded fast-path coverage (#6272).

🛠 Build & Infrastructure

• Bumped gosu in Dockerfile to address CVE-2023-28642 (#6252).
• Added license field to Rust crates (#6284).
• CI pinned to macOS 14 for consistent builds (#6314).
• Extract site config files even if site extraction is disabled (#6281).
• Moved CI coverage jobs to Hetzner (#6308).
• Improved Windows service wrapper (#6229).

👩‍💻 Developer Experience

• Added infrastructure for PostgreSQL wire protocol Golang tests (#6298).
• Enhanced test robustness for nd-array memory leak detection (#6297).
• Prevented authorization errors immediately after table creation (#6337).

🌟 New Contributors

• @mcadariu – GroupByArraySink and array aggregation functions
• @Copilot – HTTP cookie parser compatibility fix
• @shubhammenroy – PostgreSQL wire protocol Golang test integration

🔗 Full Changelog

Compare 9.1.0 → 9.1.1 on GitHub

✅ Summary

QuestDB 9.1.1 extends the database’s analytical performance and interoperability, introducing Parquet export, nanosecond ingestion, and numerous query engine optimizations.
This release strengthens QuestDB’s position as a high-performance, open, and developer-friendly time-series database for real-world workloads.

What's Changed

• feat(sql): rnd_symbol_zipfn() function - generates high cardinality symbols with Zipf/Power Law Distribution by @bluestreak01 in #6227
• fix(sql): avoid critical level logs on vectorized group by query cancellation by @puzpuzpuz in #6235
• fix(sql): overflow in materialized view refresh interval estimate calculation by @puzpuzpuz in #6233
• fix(core): potential connection leakage on Windows by @puzpuzpuz in #6243
• fix(ui): prevent overriding existing console configuration on upgrades by @jerrinot in #6240
• perf(sql): recognize data sorted by timestamp in read_parquet() SQL function by @puzpuzpuz in #6079
• fix(sql): overflow in materialized view refresh step calculation on large sample by interval by @puzpuzpuz in #6258
• fix(sql): interval constant can be applied as time intrinsics by @kafka1991 in #6263
• fix(sql): incorrect results produced by SAMPLE BY with mixed timestamp and aggregate expressions by @puzpuzpuz in #6254
• fix(sql): fix npe issue in SqlOptimiser::rewriteOrderBy by @kafka1991 in #6265
• fix(sql): fix NullPointerException in lead() window function when using bind variables by @nwoolmer in #6264
• feat(sql): add some nanos function and fix cast for date by @kafka1991 in #6247
• fix(core): fix NPE in ColumnPurgeJob by @jerrinot in #6274
• perf(sql): disable column pre-touch in parallel filter by @puzpuzpuz in #6280
• perf(sql): apply fast path for ordered and limited queries over single long column to more scenarios by @puzpuzpuz in #6272
• feat(sql): add exporting PARQUET functionality to COPY by @nwoolmer in #6008
• perf(sql): lazy merge in parallel count_distinct functions by @puzpuzpuz in #6268
• fix(sql): fix sample by negative offset by @kafka1991 in #6306
• feat(core): Java ILP client nanos precision by @glasstiger in #6220
• fix(core): fix index error after column add and truncate of non-partioned table by @ideoma in #6319
• feat(sql): optimized ASOF JOIN on single symbol key where RHS symbol is low-frequency by @mtopolnik in #6208
• fix(sql): JIT compilation on symbol column equality filters leads to incorrect query result by @puzpuzpuz in #6321
• fix(sql): parquet export erroring when writing descending ordered timestamps by @nwoolmer in #6324
• feat(sql): add last(array) function by @mcadariu in #6291
• fix(http): relax cookies parser to support HTTP 1.0 and ANSI C timestamp formats by @Copilot in #6290
• fix(sql): breaking change 💥- invalid column error returned from GROUP BY on joined tables by @puzpuzpuz in #6275
• feat(sql): require a query hint to enable rare-symbols optimization in ASOF JOIN by @mtopolnik in #6333
• test(pgwire): infrastructure ...