Skip to content

Roadmap 2024 (discussion) #58392

@alexey-milovidov

Description

@alexey-milovidov

This is ClickHouse roadmap 2024.
Descriptions and links are to be filled.

This roadmap does not cover the tasks related to infrastructure, orchestration, documentation, marketing, external integrations, drivers, etc.

See also:

Roadmap 2023: #44767
Roadmap 2022: #32513
Roadmap 2021: #17623
Roadmap 2020: link

SQL Compatibility

✔️ Enable Analyzer by default
✔️ Non-constant CASE, non-constant IN
Remove old predicate pushdown mechanics
✔️ Correlated subqueries with decorrelation
Transforming anti-join: LEFT JOIN ... WHERE ... IS NULL to NOT IN
Deriving index condition from the right-hand side of INNER JOIN
✔️ JOINs reordering and extended pushdown
✔️ Time data type

Data Storage

✔️ Userspace page cache
✔️ Adaptive mode for asynchronous inserts
✔️ Semistructured Data: Variant data type
Semistructured Data: Sharded Maps
✔️ Semistructured Data: JSON data type
Transactions for Replicated tables
✔️ Lightweight Updates v2
Uniform treatment of LowCardinality, Sparse, and Const columns
✔️ Settings to control the consistency of projections on updates
✔️ Replicated Catalog ☁️
✔️ On-disk storage for Keeper
Query cache on disk
✔️ Decoupling of object storages and metadata
✔️ Full-text indices (production readiness)
✔️ Vector search indices (production readiness)

Security, access control, and isolation

✔️ Definers (encapsulation of access control) for views
✔️ Warnings and limits on the number of database objects
Dynamic configuration of query handlers
✔️ JWT authentication ☁️
✔️ Data masking in row-level security ☁️
Secure storage for named collections ☁️
✔️ Cancellation points for long operations
✔️ Resource scheduler (continuation)

Query Processing

✔️ Parallel replicas with task callbacks (production readiness)
✔️ Parallel replicas with parallel distributed INSERT SELECT
✔️ Automatic usage of -Cluster table functions
✔️ Adaptive thresholds for data spilling on disk
✔️ Optimization with subcolumns by default

Interfaces & External Data

✔️ Support for Iceberg Data Catalog
✔️ Support for Hive-style partitioning
Explicit queries in external tables
Even simpler data upload
HTTP API for simple query construction
Unification of data lake and file-like functions

Testing & Hardening

✔️ Revive coverage
Fuzzer of data formats
Fuzzer of network protocols
✔️ Server-side AST query fuzzer
✔️ Generic fuzzer for query text
Randomization of DETACH/ATTACH in tests
Integration with SQLSmith
✔️ Embedded documentation

Experiments & Research

Cancelled: Multi-RAFT for Keeper
Cancelled: MaterializedPostgreSQL (production readiness)
✔️ SSH protocol for the server
Support for PromQL
Streaming queries
Freeform text format
Key-value data marts
Decouple of columns and buffers
✔️ Lazy reading of ranges
✔️ Instant attaching tables from backups
An object storage to borrow space from the filesystem cache
COW disks
ALTER PRIMARY KEY
Autocompletion with language models
Decentralized tables
Unique Key Constraint


The roadmap covers the top focus items for both external contributors and full-time ClickHouse employees.
The items marked with the ☁️ icon are meant for ClickHouse Cloud (proprietary).
We expect 50..80% completion of the roadmap according to the results from previous years.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions