Skip to content

[CH] Not found column: when merge spilled aggregated data from disk #8096

@lgbo-ustc

Description

@lgbo-ustc

Backend

CH (ClickHouse)

Bug description

[Expected behavior] and [actual behavior].

Caused by: org.apache.gluten.exception.GlutenException: Not found column count(1_341)_351 in block. There are only columns: col_0, col_3, col_4, col_5, col_6, multiIf(equals_262,nlp_263,equals_273,keywords_274,if_282), broadcast_right_remark#125, broadcast_right_post_id#100, equals_292, if(equals_292,1_293,0_294), if(and_296,1_297,0_298), if(and_308,1_309,0_310), if(and_313,1_314,0_315), if(and_318,1_319,0_320), if(and_323,1_324,0_325), if(and_328,1_329,0_330), if(and_333,1_334,0_335), if(and_338,1_339,0_340), 1_341, toNullable_342, toNullable_343, toNullable_344, toNullable_345, toNullable_346, toNullable_347, toNullable_348, toNullable_349, toNullable_350: While executing GraceAggregatingTransform
0. ../contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x00000000150a4a19
1. ./build/../src/Common/Exception.cpp:109: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000006ad961c
2. ../src/Common/Exception.h:111: DB::Exception::Exception(PreformattedMessage&&, int) @ 0x00000000069c56ac
3. ../src/Common/Exception.h:129: DB::Exception::Exception<String const&, String>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<String>::type>, String const&, String&&) @ 0x0000000006a6b86b
4. ./build/../src/Core/Block.cpp:313: DB::Block::getByName(String const&, bool) const @ 0x000000000f0d044e
5. ./build/../src/Interpreters/Aggregator.cpp:272: DB::Aggregator::Params::makeAggregateColumnsData(DB::Block const&) const @ 0x000000000f6d81b1
6. ./build/../src/Interpreters/Aggregator.cpp:2806: void DB::Aggregator::mergeStreamsImpl<DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef>, HashTableNoState>, DefaultHash<StringRef>, HashTableGrowerWithPrecalculation<8ul>, Allocator<true, true>>, true, true>, HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef>, HashTableNoState>, DefaultHash<StringRef>, HashTableGrowerWithPrecalculation<8ul>, Allocator<true, true>>>(DB::Block, DB::Arena*, DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef>, HashTableNoState>, DefaultHash<StringRef>, HashTableGrowerWithPrecalculation<8ul>, Allocator<true, true>>, true, true>&, HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef>, HashTableNoState>, DefaultHash<StringRef>, HashTableGrowerWithPrecalculation<8ul>, Allocator<true, true>>&, char*, DB::ColumnsHashing::LastElementCacheStats&, bool, DB::Arena*) const @ 0x000000000f776796
7. ./build/../src/Interpreters/Aggregator.cpp:2926: DB::Aggregator::mergeOnBlock(DB::Block, DB::AggregatedDataVariants&, bool&, std::atomic<bool>&) const @ 0x000000000f6e6bea
8. ./build/../utils/extern-local-engine/Operator/GraceAggregatingTransform.cpp:510: local_engine::GraceAggregatingTransform::mergeOneBlock(DB::Block const&, bool) @ 0x00000000070a1fbf
9. ./build/../utils/extern-local-engine/Operator/GraceAggregatingTransform.cpp:400: local_engine::GraceAggregatingTransform::prepareBucketOutputBlocks(unsigned long) @ 0x00000000070a2c39
10. ./build/../utils/extern-local-engine/Operator/GraceAggregatingTransform.cpp:158: local_engine::GraceAggregatingTransform::work() @ 0x00000000070a1168
11. ./build/../src/Processors/Executors/ExecutionThreadContext.cpp:47: DB::ExecutionThreadContext::executeTask() @ 0x00000000119d3c22
12. ./build/../src/Processors/Executors/PipelineExecutor.cpp:289: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x00000000119c8f1f
13. ./build/../src/Processors/Executors/PipelineExecutor.cpp:255: DB::PipelineExecutor::executeImpl(unsigned long, bool) @ 0x00000000119c86dd
14. ./build/../src/Processors/Executors/PipelineExecutor.cpp:126: DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000119c84cb
15. ./build/../utils/extern-local-engine/Parser/LocalExecutor.cpp:130: local_engine::LocalExecutor::execute() @ 0x0000000006eb7402

Spark version

None

Spark configurations

No response

System information

No response

Relevant logs

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingtriage

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions