[GLUTEN-8846][CH] [Part 3] Add benchmark for Icerberg Delete#9192
[GLUTEN-8846][CH] [Part 3] Add benchmark for Icerberg Delete#9192baibaichen merged 4 commits intoapache:mainfrom
Conversation
…em' data from the local system.
|
Run Gluten Clickhouse CI on x86 |
There was a problem hiding this comment.
Copilot reviewed 11 out of 25 changed files in this pull request and generated no comments.
Files not reviewed (14)
- backends-clickhouse/src/test/scala/org/apache/gluten/execution/GlutenClickHouseTPCHAbstractSuite.scala: Language not supported
- backends-clickhouse/src/test/scala/org/apache/gluten/execution/tpch/GlutenClickHouseHDFSSuite.scala: Language not supported
- cpp-ch/local-engine/Common/BlockTypeUtils.cpp: Language not supported
- cpp-ch/local-engine/Common/BlockTypeUtils.h: Language not supported
- cpp-ch/local-engine/Common/PlanUtil.cpp: Language not supported
- cpp-ch/local-engine/Parser/FunctionExecutor.cpp: Language not supported
- cpp-ch/local-engine/Parser/RelParsers/ReadRelParser.cpp: Language not supported
- cpp-ch/local-engine/Parser/RelParsers/WriteRelParser.cpp: Language not supported
- cpp-ch/local-engine/Storages/SubstraitSource/CMakeLists.txt: Language not supported
- cpp-ch/local-engine/Storages/SubstraitSource/FileReader.cpp: Language not supported
- cpp-ch/local-engine/Storages/SubstraitSource/Iceberg/EqualityDeleteFileReader.cpp: Language not supported
- cpp-ch/local-engine/Storages/SubstraitSource/Iceberg/IcebergReader.cpp: Language not supported
- cpp-ch/local-engine/Storages/SubstraitSource/Iceberg/PositionalDeleteFileReader.cpp: Language not supported
- cpp-ch/local-engine/Storages/SubstraitSource/ParquetFormatFile.cpp: Language not supported
|
@CodiumAI-Agent /review |
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
|
@CodiumAI-Agent /improve |
PR Code Suggestions ✨
|
* [GLUTEN-1632][CH]Daily Update Clickhouse Version (20250405) * Fix Build due to ClickHouse/ClickHouse#78515 * remove temp code due to #9192 * Fix test fail due to ClickHouse/ClickHouse#78022 --------- Co-authored-by: kyligence-git <[email protected]> Co-authored-by: Chang chen <[email protected]>
What changes were proposed in this pull request?
(Fixes: #8846)
Summary of Changes
This commit primarily:
Key Changes
Directory Structure and Naming
icebergdirectory toIcebergwith proper capitalizationCode Improvements
begion→beginin BlockTypeUtils.hNormalFileReaderconstructorstd::ranges::fold_leftNew Functionality: Added code for benchmarking Iceberg read operations with equality deletes and position deletes
These two benchmark functions (
BM_IcebergReadWithPositionDeletesandBM_IcebergReadWithEqualityDeletes) are designed to evaluate the performance of ClickHouse when integrated with Apache Iceberg delete operations. Supports parameterized testing with different deletion ratios (0% to 100%)Position Delete Performance (BM_IcebergReadWithPositionDeletes)
Position delete tests show performance decreases as deletion ratio increases, but the growth is relatively slow:
This indicates that for position deletes, performance degradation has a near-linear relationship with deletion ratio, with processing time increasing approximately 8.8 times from 0 to 100% deletion ratio.
Equality Delete Performance (BM_IcebergReadWithEqualityDeletes)
Equality delete tests demonstrate a dramatic performance decline:
Equality delete performance degradation shows exponential growth, with processing time increasing approximately 264 times from 0 to 50% deletion ratio.
Analysis
There is only one hotpot, i.e.
DB::Set::executeImplCasewhich is part of the logic for checking if elements in a set of columns belong to a predefined set, and supporting both positive and negative set operations.How was this patch tested?
Existed UTs