Skip to content

Crash Bug - Segmentation fault - Address: NULL pointer. Access: read. Unknown si_code. #45590

@mignaulo

Description

@mignaulo

Crash bug on ClickHouse server version 22.12.3.5 (official build), running on Ubuntu 22.04

2023.01.24 20:15:07.396001 [ 419381 ] {} <Fatal> BaseDaemon: ########################################
2023.01.24 20:15:07.396083 [ 419381 ] {} <Fatal> BaseDaemon: (version 22.12.3.5 (official build), build id: 1FC9EF37AB97BF195F9DF1C7D44CEC3DCC837FAD) (from thread 409241) (query_id: a86fef4a-d7fe-4046-8d30-49f6e5bb155b) (query:  with tt as ( Select cast(toStartOfInterval(timestamp, INTERVAL 1 day) as Date) as dd, count() as samples from db.table_live_v2 group by dd having dd >= toDate(now())-100 ), tt2 as ( Select dd, samples from tt union distinct Select toDate(now())-1, ifnull((select samples from tt where dd = toDate(now())-1),0) as samples ) Select dd, samples from tt2 order by dd WITH FILL STEP 1 ) Received signal Segmentation fault (11)
2023.01.24 20:15:07.396108 [ 419381 ] {} <Fatal> BaseDaemon: Address: NULL pointer. Access: read. Unknown si_code.
2023.01.24 20:15:07.396136 [ 419381 ] {} <Fatal> BaseDaemon: Stack trace: 0x13942618 0x1394188f 0x13908e88 0x139079ba 0x142e9b8a 0x142fb9d4 0x142efbaf 0x142eec6d 0x143896b4 0x142fbd64 0x142efbaf 0x142eec6d 0x143892e5 0x1438a2fe 0x14692bb0 0x1468fdad 0x1538858f 0x1539be59 0x1821ac34 0x1821c5db 0x183ab6c7 0x183a90fd 0x7f14cc002b43 0x7f14cc094a00
2023.01.24 20:15:07.396193 [ 419381 ] {} <Fatal> BaseDaemon: 2. ? @ 0x13942618 in /usr/bin/clickhouse
2023.01.24 20:15:07.396233 [ 419381 ] {} <Fatal> BaseDaemon: 3. ? @ 0x1394188f in /usr/bin/clickhouse
2023.01.24 20:15:07.396256 [ 419381 ] {} <Fatal> BaseDaemon: 4. ? @ 0x13908e88 in /usr/bin/clickhouse
2023.01.24 20:15:07.396298 [ 419381 ] {} <Fatal> BaseDaemon: 5. DB::Context::Context(DB::Context const&) @ 0x139079ba in /usr/bin/clickhouse
2023.01.24 20:15:07.396337 [ 419381 ] {} <Fatal> BaseDaemon: 6. ? @ 0x142e9b8a in /usr/bin/clickhouse
2023.01.24 20:15:07.396372 [ 419381 ] {} <Fatal> BaseDaemon: 7. DB::InterpreterSelectQuery::executeFetchColumns(DB::QueryProcessingStage::Enum, DB::QueryPlan&) @ 0x142fb9d4 in /usr/bin/clickhouse
2023.01.24 20:15:07.396423 [ 419381 ] {} <Fatal> BaseDaemon: 8. DB::InterpreterSelectQuery::executeImpl(DB::QueryPlan&, std::__1::optional<DB::Pipe>) @ 0x142efbaf in /usr/bin/clickhouse
2023.01.24 20:15:07.396449 [ 419381 ] {} <Fatal> BaseDaemon: 9. DB::InterpreterSelectQuery::buildQueryPlan(DB::QueryPlan&) @ 0x142eec6d in /usr/bin/clickhouse
2023.01.24 20:15:07.396479 [ 419381 ] {} <Fatal> BaseDaemon: 10. DB::InterpreterSelectWithUnionQuery::buildQueryPlan(DB::QueryPlan&) @ 0x143896b4 in /usr/bin/clickhouse
2023.01.24 20:15:07.396505 [ 419381 ] {} <Fatal> BaseDaemon: 11. DB::InterpreterSelectQuery::executeFetchColumns(DB::QueryProcessingStage::Enum, DB::QueryPlan&) @ 0x142fbd64 in /usr/bin/clickhouse
2023.01.24 20:15:07.396545 [ 419381 ] {} <Fatal> BaseDaemon: 12. DB::InterpreterSelectQuery::executeImpl(DB::QueryPlan&, std::__1::optional<DB::Pipe>) @ 0x142efbaf in /usr/bin/clickhouse
2023.01.24 20:15:07.396573 [ 419381 ] {} <Fatal> BaseDaemon: 13. DB::InterpreterSelectQuery::buildQueryPlan(DB::QueryPlan&) @ 0x142eec6d in /usr/bin/clickhouse
2023.01.24 20:15:07.396601 [ 419381 ] {} <Fatal> BaseDaemon: 14. DB::InterpreterSelectWithUnionQuery::buildQueryPlan(DB::QueryPlan&) @ 0x143892e5 in /usr/bin/clickhouse
2023.01.24 20:15:07.396627 [ 419381 ] {} <Fatal> BaseDaemon: 15. DB::InterpreterSelectWithUnionQuery::execute() @ 0x1438a2fe in /usr/bin/clickhouse
2023.01.24 20:15:07.396657 [ 419381 ] {} <Fatal> BaseDaemon: 16. ? @ 0x14692bb0 in /usr/bin/clickhouse
2023.01.24 20:15:07.396687 [ 419381 ] {} <Fatal> BaseDaemon: 17. DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x1468fdad in /usr/bin/clickhouse
2023.01.24 20:15:07.396719 [ 419381 ] {} <Fatal> BaseDaemon: 18. DB::TCPHandler::runImpl() @ 0x1538858f in /usr/bin/clickhouse
2023.01.24 20:15:07.396742 [ 419381 ] {} <Fatal> BaseDaemon: 19. DB::TCPHandler::run() @ 0x1539be59 in /usr/bin/clickhouse
2023.01.24 20:15:07.396777 [ 419381 ] {} <Fatal> BaseDaemon: 20. Poco::Net::TCPServerConnection::start() @ 0x1821ac34 in /usr/bin/clickhouse
2023.01.24 20:15:07.396806 [ 419381 ] {} <Fatal> BaseDaemon: 21. Poco::Net::TCPServerDispatcher::run() @ 0x1821c5db in /usr/bin/clickhouse
2023.01.24 20:15:07.396837 [ 419381 ] {} <Fatal> BaseDaemon: 22. Poco::PooledThread::run() @ 0x183ab6c7 in /usr/bin/clickhouse
2023.01.24 20:15:07.396900 [ 419381 ] {} <Fatal> BaseDaemon: 23. Poco::ThreadImpl::runnableEntry(void*) @ 0x183a90fd in /usr/bin/clickhouse
2023.01.24 20:15:07.396925 [ 419381 ] {} <Fatal> BaseDaemon: 24. ? @ 0x7f14cc002b43 in ?
2023.01.24 20:15:07.396955 [ 419381 ] {} <Fatal> BaseDaemon: 25. ? @ 0x7f14cc094a00 in ?
2023.01.24 20:15:07.547764 [ 419381 ] {} <Fatal> BaseDaemon: Integrity check of the executable successfully passed (checksum: 707A227274554878C01CAF89F747CA21)

The query attached to the crash report is quite simple:

with tt as ( 
    Select cast(toStartOfInterval(timestamp, INTERVAL 1 day) as Date) as dd, count() as samples 
    from db.table_live_v2 
    group by dd having dd >= toDate(now())-100 
    ),
tt2 as ( 
    Select dd, samples from tt 
    union distinct 
    Select toDate(now())-1, ifnull((select samples from tt where dd = toDate(now())-1),0) as samples 
) 
Select dd, samples from tt2 order by dd WITH FILL STEP 1

Rerunning the query about 10 times in a row, I was able to crash Clickhouse again, with a different error

2023.01.24 22:18:19.532146 [ 425314 ] <Fatal> BaseDaemon: (version 22.12.3.5 (official build), build id: 1FC9EF37AB97BF195F9DF1C7D44CEC3DCC837FAD) (from thread 419389) (query_id: 8ecb919e-26ca-459d-9158-43e30146c529) (query: with tt as ( Select cast(toStartOfInterval(timestamp, INTERVAL 1 day) as Date) as dd, count() as samples from db.table_live_v2 group by dd having dd >= toDate(now())-100 ), tt2 as ( Select dd, samples from tt union distinct Select toDate(now())-1, ifnull((select samples from tt where dd = toDate(now())-1),0) as samples ) Select dd, samples from tt2 order by dd WITH FILL STEP 1) Received signal Segmentation fault (11)
2023.01.24 22:18:19.532173 [ 425314 ] <Fatal> BaseDaemon: Address: NULL pointer. Access: read. Address not mapped to object.
2023.01.24 22:18:19.532201 [ 425314 ] <Fatal> BaseDaemon: Stack trace: 0x142f6242 0x14302d18 0x142f1b6d 0x142eec6d 0x143892e5 0x1438a2fe 0x14692bb0 0x1468fdad 0x1538858f ac34 0x1821c5db 0x183ab6c7 0x183a90fd 0x7f06418fab43 0x7f064198ca00
2023.01.24 22:18:19.532256 [ 425314 ] <Fatal> BaseDaemon: 2. DB::InterpreterSelectQuery::getSortDescription(DB::ASTSelectQuery const&, std::__1::shared_ptr<DB::Context 0x142f6242 in /usr/bin/clickhouse
2023.01.24 22:18:19.532280 [ 425314 ] <Fatal> BaseDaemon: 3. DB::InterpreterSelectQuery::executeOrder(DB::QueryPlan&, std::__1::shared_ptr<DB::InputOrderInfo const>) @ r/bin/clickhouse
2023.01.24 22:18:19.532293 [ 425314 ] <Fatal> BaseDaemon: 4. DB::InterpreterSelectQuery::executeImpl(DB::QueryPlan&, std::__1::optional<DB::Pipe>) @ 0x142f1b6d in /usr/
2023.01.24 22:18:19.532312 [ 425314 ] <Fatal> BaseDaemon: 5. DB::InterpreterSelectQuery::buildQueryPlan(DB::QueryPlan&) @ 0x142eec6d in /usr/bin/clickhouse
2023.01.24 22:18:19.532330 [ 425314 ] <Fatal> BaseDaemon: 6. DB::InterpreterSelectWithUnionQuery::buildQueryPlan(DB::QueryPlan&) @ 0x143892e5 in /usr/bin/clickhouse
2023.01.24 22:18:19.532361 [ 425314 ] <Fatal> BaseDaemon: 7. DB::InterpreterSelectWithUnionQuery::execute() @ 0x1438a2fe in /usr/bin/clickhouse
2023.01.24 22:18:19.532379 [ 425314 ] <Fatal> BaseDaemon: 8. ? @ 0x14692bb0 in /usr/bin/clickhouse
2023.01.24 22:18:19.532415 [ 425314 ] <Fatal> BaseDaemon: 9. DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> :shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x1468fdad in /usr/bin/clickhouse
2023.01.24 22:18:19.532440 [ 425314 ] <Fatal> BaseDaemon: 10. DB::TCPHandler::runImpl() @ 0x1538858f in /usr/bin/clickhouse
2023.01.24 22:18:19.532457 [ 425314 ] <Fatal> BaseDaemon: 11. DB::TCPHandler::run() @ 0x1539be59 in /usr/bin/clickhouse
2023.01.24 22:18:19.532479 [ 425314 ] <Fatal> BaseDaemon: 12. Poco::Net::TCPServerConnection::start() @ 0x1821ac34 in /usr/bin/clickhouse
2023.01.24 22:18:19.532497 [ 425314 ] <Fatal> BaseDaemon: 13. Poco::Net::TCPServerDispatcher::run() @ 0x1821c5db in /usr/bin/clickhouse
2023.01.24 22:18:19.532524 [ 425314 ] <Fatal> BaseDaemon: 14. Poco::PooledThread::run() @ 0x183ab6c7 in /usr/bin/clickhouse
2023.01.24 22:18:19.532558 [ 425314 ] <Fatal> BaseDaemon: 15. Poco::ThreadImpl::runnableEntry(void*) @ 0x183a90fd in /usr/bin/clickhouse
2023.01.24 22:18:19.532577 [ 425314 ] <Fatal> BaseDaemon: 16. ? @ 0x7f06418fab43 in ?
2023.01.24 22:18:19.532593 [ 425314 ] <Fatal> BaseDaemon: 17. ? @ 0x7f064198ca00 in ?
2023.01.24 22:18:19.683348 [ 425314 ] <Fatal> BaseDaemon: Integrity check of the executable successfully passed (checksum: 707A227274554878C01CAF89F747CA21)

Thank you for all your help.

Metadata

Metadata

Labels

bugConfirmed user-visible misbehaviour in official releasecomp-projectionsProjections: precomputed layouts for accelerating queries (projection parts, maintenance).crashCrash / segfault / abortmajor

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions