Blog
2026
Faster DuckDB Queries with Vortex
With the release of DuckDB 2.0, we look at how the Vortex core extension speeds up queries by computing over compressed data, and how to read and write Vortex files from DuckDB.
What Vortex Has to Prove Before Iceberg Adds a Fourth File Format
The Vortex proposal split into criteria and candidate. What field IDs, statistics, governance, and ecosystem support a new Iceberg file format must prove.
Statistics are aggregates
Min-max statistics are aggregates computed at write time. Treated that way, they can answer queries, not just skip data.
Fusing GPU Kernels with Dynamic Dispatch
GPU decompression at 2 TB/s with Fused Dynamic Dispatch. How Vortex doubles throughput by fusing CUDA kernels.
From S3 to GPU in One Copy
We optimized Vortex so it can stream data from object storage into GPU memory fast enough to saturate a 100 Gbps network link. The first step to get this performance is to make sure each byte is only copied once between the kernel and the GPU.
Vortex: One Format for Any Shape
How Vortex keeps logical types, physical encodings, and storage layouts open to extension so one columnar format can fit any shape of data.
ParquetReader Now Supports Vortex Files
ParquetReader now supports Vortex files. Learn what the Vortex columnar format is and how to open, inspect, and convert Vortex data to CSV, JSON, or Parquet.
The Storage Layer for the Composable Data Stack
A composable query engine deserves a composable storage layer. Build the format your workload needs without rebuilding the storage stack.
Compile or Prefilter? Matching LIKE over Compressed Strings
Two teams independently built SQL LIKE over FSST-compressed strings: Vortex prefilters candidates, TU Munich compiles the pattern. One experiment across seven x86 generations maps where each wins, where they cross, and why the two hot loops belong in one system.
Pattern Matching on Compressed Strings without Decompression
Evaluating SQL LIKE directly on FSST-compressed strings: a symbol-aware automaton and a SIMD prefilter let Vortex match without decompressing the column first.
My Journey Optimizing a CUDA Kernel with Polar Signals
A storage engineer's descent into warp stalls, memory throttles, and aligned stores
AI Datasets Are Never Done. So We Built Loon.
Loon is a new storage engine for Milvus 3.0 and Zilliz Vector Lakebase, built to manage evolving vector datasets with ColumnGroups, row ID alignment, and Manifests.
We built SmithDB, the data layer for agent observability
Introducing SmithDB: LangSmith's purpose-built distributed database for agent observability, delivering up to 12x faster performance with full portability.
Vortex: The Columnar Format for Data Workloads
Learn how Spice AI uses the Vortex columnar format in Cayenne to speed data-intensive workloads, reduce memory use, and scale concurrent ingestion and queries.
Have your cake and decompress it too
How Vortex uses BtrBlocks-style codec selection to beat Parquet+ZSTD on both size and speed.
Introducing the Apache Iceberg File Format API
The Apache Iceberg community is excited to announce the finalization of the File Format API, a major architectural milestone that makes file formats pluggable, consistent, and engine‑agnostic across the Iceberg Java codebase.
December & January Bulletin
Lazy array evaluation shipped in 0.57.0, CUDA kernels for decoding and filtering landed, and DataFusion 52 support merged.
Announcing Vortex Support in DuckDB
Vortex is a new columnar file format with a very promising design. SpiralDB and DuckLabs have partnered to give you a very fast experience while reading and writing Vortex files!
Databases in 2025: A Year in Review
The world tried to kill Andy off but he had to stay alive to talk about what happened with databases in 2025.
2025
Introducing the Spice Cayenne Data Accelerator
Spice Cayenne is the next-generation Spice.ai data accelerator built on the Vortex columnar format for high-scale, low-latency data lake acceleration workloads.
November Bulletin
Spice AI launched Cayenne on Vortex, Polar Signals replaced Parquet with Vortex, and Vortex became a DuckDB core extension.
Great Lakes: A Delta Lake Fork
Our updates to Delta Lake Table Format
Questioning an Interface: From Parquet to Vortex
Breaking free from the shackles of interface-imposed performance limitations
October Bulletin
An experimental GPU scan landed, the DataFusion integration pushed filters into struct fields, and DuckDB 1.4.2 made Vortex a core extension.
Exploring the Evolving File Format Landscape in AI Era: Parquet, Lance, Nimble and Vortex And What It Means for Apache Iceberg
Formats like Lance, Nimble, and Vortex show that innovation is accelerating. Each one introduces new ideas about how to structure, index, and access data. But none of them exist in isolation. They need to integrate with engines, catalogs, and governance layers to be truly useful.
September Bulletin #2
We released 0.54.0, published the Operators RFC, added FastLanes-based RLE, and shipped the extension for DuckDB 1.4.
Is Parquet becoming the bottleneck? Why new storage formats are emerging in 2025 (Lance, Vortex, and more)
Parquet gave data lakes a common language: columnar layout, good compression, and fast scans.
September Bulletin
We added a Ray datasource, the FixedSizeList type, list support for DuckDB, and the TPC-DS and StatPopGen benchmarks.
Welcome to the Vortex Community Blog
Introducing a new blog to share work happening with and in the Vortex ecosystem
Vortex: a Linux Foundation Project
Building the Future of Open Source, Columnar Storage
Vortex on Ice
Using Vortex to accelerate Apache Iceberg queries up to 4x
Towards Vortex 1.0
Preparing to launch Vortex 1.0 and the 2025.05 Edition
What if we just didn’t decompress it?
How push-down compute helps Vortex run faster and decompress less
Zone maps, or “queries go brrr”
Waiting sucks. Let's use zone maps to move fast and read less.
Data Layouts: Where Bytes Find Their Forever Home
How Vortex tackles data layout challenges across analytics and machine learning workflows.
Logical vs Physical data types
The rationale behind the Vortex logical type system.