Full Decimal type support in Parquet read/write #104
Description
Summary
Complete Parquet support for all QuestDB decimal types, enabling high-precision numeric data to be read from and written to Parquet format without loss of precision.
Supported Types
| Type | Precision | Use Case |
|---|---|---|
| DECIMAL8 | Up to 2 digits | Compact small decimals |
| DECIMAL16 | Up to 4 digits | Small financial values |
| DECIMAL32 | Up to 9 digits | Standard precision |
| DECIMAL64 | Up to 18 digits | High precision |
| DECIMAL128 | Up to 38 digits | Full financial precision |
| DECIMAL256 | Up to 76 digits | Ultra-high precision |
Features
- Read support — Import decimal columns from external Parquet files
- Write support — Export decimal columns to Parquet format
- Parquet-spec compliant — Fixed-length byte arrays in big-endian format per specification
- Round-trip safe — Native → Parquet → Native without precision loss
Benefits
- Parquet interoperability — Exchange decimal-precise data with data lakes and analytics tools
- End-to-end precision — No loss when archiving to Parquet via storage policies
- Ecosystem compatibility — Parquet Decimal type widely supported (Spark, DuckDB, Pandas, etc.)
- Flexible precision — Choose the right decimal size for your use case
Use Cases
- Financial data archival with exact precision preserved
- Data lake exports for compliance and reporting
- Integration with downstream analytics pipelines
- Importing decimal data from external Parquet sources
Parent feature
Sub-issue of #62 (Native Apache Parquet partition format)