-
Notifications
You must be signed in to change notification settings - Fork 8.3k
Compression for Memory tables #20093
Description
Add very simple in-memory compression interface that works directly for columns.
Add method IColumn::compress. It returns either:
CompressedColumnPtrthat isshared_ptrofICompressedColumn, that has the only methodvirtual ColumnPtr decompress();std::function<ColumnPtr()>- a function object to get decompressed column back;
This method will take no parameters for compression but it can be extended later.
It should compress the whole buffers of columns as a single block with lz4. The usual compression interface with Read/Write buffers, support for checksums, etc. is not used.
Add support for SETTINGS clause for Memory table with the setting compress = 0|1.
The hypothesis is that compressed Memory tables will work faster on SELECTs by using less memory bandwidth.
Actually we already confirmed this hypothesis with MergeTree tables: they are compressed by default and it works faster than with disabled compression even if data is in memory and disk is not used.