Skip to content

[RFC] Intercept malloc/free and other C-style functions for memory management#84082

Merged
nikitamikhaylov merged 24 commits intomasterfrom
everything-is-on-track
Aug 6, 2025
Merged

[RFC] Intercept malloc/free and other C-style functions for memory management#84082
nikitamikhaylov merged 24 commits intomasterfrom
everything-is-on-track

Conversation

@nikitamikhaylov
Copy link
Copy Markdown
Member

@nikitamikhaylov nikitamikhaylov commented Jul 20, 2025 •
edited
Loading

This is an open wound in ClickHouse's memory management as all the allocations from external libraries through malloc/calloc/posix_memalign are not visible for the ClickHouse memory tracker.
There were previous attempt to fix it for the most annoying places, for example: #27252 and #63584, however the solution is not ideal and there are still some problems: #83165

This PR tries to utilize the linker's --wrap command that replaces the symbols at link time. It seems working, but I need to clarify certain things.

Cc @azat @alexey-milovidov

Changelog category (leave one):

  • Improvement

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

All the allocations done by external libraries are now visible to ClickHouse's memory tracker and accounted properly. This may result in "increased" reported memory usage for certain queries or failures with MEMORY_LIMIT_EXCEEDED.

Documentation entry for user-facing changes

  • Documentation is written (mandatory for new features)

@clickhouse-gh
Copy link
Copy Markdown
Contributor

clickhouse-gh bot commented Jul 20, 2025 •
edited
Loading

Workflow [PR], commit [378ece1]

Summary: ❌

job_name test_name status info comment
Stateless tests (arm_binary, parallel) failure
00601_kill_running_query FAIL
Stress test (amd_tsan) failure
Server died FAIL
Hung check failed, possible deadlock found (see hung_check.log) FAIL
Killed by signal (in clickhouse-server.log) FAIL
Fatal message in clickhouse-server.log (see fatal_messages.txt) FAIL
Killed by signal (output files) FAIL
Found signal in gdb.log FAIL
Stress test (amd_ubsan) failure
Server died FAIL
Hung check failed, possible deadlock found (see hung_check.log) FAIL
Killed by signal (in clickhouse-server.log) FAIL
Fatal message in clickhouse-server.log (see fatal_messages.txt) FAIL
Killed by signal (output files) FAIL
Found signal in gdb.log FAIL

@clickhouse-gh clickhouse-gh bot added pr-improvement Pull request with some product improvements submodule changed At least one submodule changed in this PR. labels Jul 20, 2025
nikitamikhaylov
nikitamikhaylov commented Jul 20, 2025
View reviewed changes
nikitamikhaylov
nikitamikhaylov commented Jul 20, 2025
View reviewed changes
@nikitamikhaylov nikitamikhaylov force-pushed the everything-is-on-track branch from d5f7c10 to 9e2c239 Compare July 20, 2025 20:32
@nikitamikhaylov nikitamikhaylov force-pushed the everything-is-on-track branch from cc40257 to ea664f6 Compare July 20, 2025 22:37
azat
azat reviewed Jul 21, 2025
View reviewed changes
Copy link
Copy Markdown
Member

@azat azat left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to use __real_* in src/Common/FiberStack.cpp as well.

It will be interesting to look at performance results (one of issues that could arise is that we will call MemoryTracker::allocImpl on each mallocs, due to lack of ThreadStatus somewhere in third party threads, i.e. librdkafka or possible delta-kernel-rs, and we don't have perf tests for this, so we need to do some manual testing)

Also I'm not sure that it is OK to do this for sanitizers, but CI will show...

And we need to make 100% sure that there will be no double accounting

@azat azat self-assigned this Jul 21, 2025
@nikitamikhaylov nikitamikhaylov added the pr-performance Pull request with some performance improvements label Jul 22, 2025
@nikitamikhaylov nikitamikhaylov force-pushed the everything-is-on-track branch from 6464fa6 to bb3c759 Compare July 22, 2025 22:57
@clickhouse-gh clickhouse-gh bot removed the pr-performance Pull request with some performance improvements label Jul 22, 2025
@nikitamikhaylov nikitamikhaylov requested a review from azat July 22, 2025 22:57
@nikitamikhaylov nikitamikhaylov force-pushed the everything-is-on-track branch from bb3c759 to 88e5fd5 Compare July 23, 2025 12:07
@nikitamikhaylov
Copy link
Copy Markdown
Member Author

nikitamikhaylov commented Jul 23, 2025

@azat it seems there is no significant diff in performance. 1, 2, 3.

@azat
Copy link
Copy Markdown
Member

azat commented Jul 24, 2025

one of issues that could arise is that we will call MemoryTracker::allocImpl on each mallocs, due to lack of ThreadStatus somewhere in third party threads, i.e. librdkafka or possible delta-kernel-rs, and we don't have perf tests for this, so we need to do some manual testing

Funny, Kafka storage already creates ThreadStatus for each thread, and even though it was useless (since we do not track C allocations before), it is ready for this change

But, delta-kernel-rs can be affected, not sure how it is used, but, if it creates threads and do allocations from it, they may became slower. Can you verify this?

azat
azat reviewed Jul 24, 2025
View reviewed changes
azat
azat reviewed Jul 24, 2025
View reviewed changes
@azat
Copy link
Copy Markdown
Member

azat commented Jul 24, 2025

Also, let's add a unit test for malloc/free and ensure that MemoryTracking metric will reflect the allocations

@nikitamikhaylov
Copy link
Copy Markdown
Member Author

nikitamikhaylov commented Jul 24, 2025 •
edited
Loading

About delta-kernel-rs - I do see the MemoryTracker::allocImpl and MemoryTracker::free in perf top when running this query in clickhouse-local. Note, that each time I had a debug build.

SELECT
    count() AS installs,
    project
FROM deltaLakeCluster(default, 's3://clickhouse-datalake-demo/data/pypi/2023/delta/part', '', '', SETTINGS allow_experimental_delta_kernel_rs = 1)
WHERE event_date = '2023-04-03'
GROUP BY project
ORDER BY installs DESC
LIMIT 10
SETTINGS use_hive_partitioning = false

Current PR:

  22,50%  clickhouse        [.] memcpy
   3,90%  clickhouse        [.] _$LT$delta_kernel..engine..default..json..DefaultJsonHandler$LT$E$GT$$u20$as$u20$delta_kernel..JsonHandler$GT$::read_json_files::_$u7b$$u7b$clos
   2,40%  clickhouse        [.] arrow_json::reader::tape::TapeDecoder::finish
   1,96%  clickhouse        [.] MemoryTracker::allocImpl(long, bool, MemoryTracker*, double)
   1,83%  clickhouse        [.] std::sync::mpmc::array::Channel<T>::send
   1,82%  clickhouse        [.] delta_kernel::engine::default::json::JsonOpener::open::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::hce67c46c22d93036
   1,80%  clickhouse        [.] <futures_util::stream::try_stream::try_unfold::TryUnfold<T,F,Fut> as futures_core::stream::Stream>::poll_next
   1,77%  clickhouse        [.] std::sync::mpmc::array::Channel<T>::start_send
   1,77%  clickhouse        [.] core::option::Option<T>::map
   1,65%  clickhouse        [.] std::sync::mpmc::Sender<T>::send
   1,62%  clickhouse        [.] core::fmt::write
   1,61%  clickhouse        [.] alloc::fmt::format::format_inner
   1,60%  clickhouse        [.] <futures_util::stream::stream::map::Map<St,F> as futures_core::stream::Stream>::poll_next
   1,55%  clickhouse        [.] MemoryTracker::free(long, double)
   1,48%  clickhouse        [.] core::result::Result<core::option::Option<T>,E>::transpose
   1,47%  clickhouse        [.] arrow_json::reader::Decoder::flush
   1,42%  clickhouse        [.] <futures_util::stream::try_stream::try_flatten::TryFlatten<St> as futures_core::stream::Stream>::poll_next
   1,39%  clickhouse        [.] core::sync::atomic::AtomicUsize::load
   1,33%  clickhouse        [.] <futures_util::stream::stream::map::Map<St,F> as futures_core::stream::Stream>::poll_next
   1,24%  clickhouse        [.] <futures_util::stream::stream::map::Map<St,F> as futures_core::stream::Stream>::poll_next
   1,17%  clickhouse        [.] CurrentMemoryTracker::allocImpl(long, bool)
   1,14%  clickhouse        [.] std::sync::mpmc::Sender$LT$T$GT$::send::_$u7b$$u7b$closure$u7d$$u7d$::h89b262351b2aeb3c
   1,14%  clickhouse        [.] <core::task::poll::Poll<core::option::Option<core::result::Result<T,E>>> as core::ops::try_trait::Try>::branch
   1,12%  clickhouse        [.] futures_util::stream::stream::StreamExt::poll_next_unpin
   1,04%  clickhouse        [.] core::ptr::drop_in_place<core::result::Result<alloc::boxed::Box<dyn delta_kernel::engine_data::EngineData>,delta_kernel::error::Error>>
   1,04%  clickhouse        [.] <alloc::string::String as core::fmt::Write>::write_str
   1,02%  clickhouse        [.] <futures_util::fns::MapErrFn<F> as futures_util::fns::FnMut1<core::result::Result<T,E>>>::call_mut
   0,94%  clickhouse        [.] <futures_util::stream::stream::next::Next<St> as core::future::future::Future>::poll
   0,91%  clickhouse        [.] <core::pin::Pin<P> as futures_core::stream::Stream>::poll_next
   0,90%  clickhouse        [.] <S as futures_core::stream::TryStream>::try_poll_next
   0,86%  clickhouse        [.] <S as futures_core::stream::TryStream>::try_poll_next
   0,83%  clickhouse        [.] core::ptr::drop_in_place<arrow_schema::error::ArrowError>
   0,82%  clickhouse        [.] alloc::fmt::format
   0,80%  clickhouse        [.] _$LT$futures_util..stream..stream..map..Map$LT$St$C$F$GT$$u20$as$u20$futures_core..stream..Stream$GT$::poll_next::_$u7b$$u7b$closure$u7d$$u7d$::
   0,79%  clickhouse        [.] core::sync::atomic::AtomicBool::load
   0,76%  clickhouse        [.] <T as futures_util::fns::FnMut1<A>>::call_mut
   0,73%  clickhouse        [.] core::ops::function::FnOnce::call_once
   0,72%  clickhouse        [.] <S as futures_core::stream::TryStream>::try_poll_next
   0,70%  clickhouse        [.] core::slice::raw::from_raw_parts::precondition_check
   0,70%  clickhouse        [.] __wrap_free
   0,68%  clickhouse        [.] core::sync::atomic::atomic_load
   0,68%  clickhouse        [.] <core::pin::Pin<P> as futures_core::stream::Stream>::poll_next
   0,68%  clickhouse        [.] <core::task::poll::Poll<core::option::Option<core::result::Result<T,F>>> as core::ops::try_trait::FromResidual<core::result::Result<core::conver
   0,67%  clickhouse        [.] <futures_util::stream::try_stream::into_stream::IntoStream<St> as futures_core::stream::Stream>::poll_next
   0,66%  clickhouse        [.] core::ptr::drop_in_place<std::sync::mpsc::SendError<core::result::Result<alloc::boxed::Box<dyn delta_kernel::engine_data::EngineData>,delta_kern
   0,60%  clickhouse        [.] core::option::Option<T>::map
   0,59%  clickhouse        [.] core::result::Result<T,E>::map_err
   0,57%  clickhouse        [.] delta_kernel::error::Error::with_backtrace
   0,57%  clickhouse        [.] alloc::fmt::format::_$u7b$$u7b$closure$u7d$$u7d$::h0511f2b35bbdd50a
   0,56%  clickhouse        [.] <S as futures_core::stream::TryStream>::try_poll_next
   0,54%  clickhouse        [.] core::ptr::drop_in_place<delta_kernel::error::Error>
   0,53%  clickhouse        [.] core::option::Option<T>::map
   0,52%  clickhouse        [.] std::sync::mpsc::SyncSender<T>::send
   0,51%  clickhouse        [.] free
   0,51%  clickhouse        [.] core::ptr::drop_in_place<std::backtrace::Backtrace>
   0,50%  clickhouse        [.] core::cmp::PartialOrd::le
   0,49%  clickhouse        [.] <futures_util::stream::try_stream::MapOk<St,F> as futures_core::stream::Stream>::poll_next

On master:

 25,56%  clickhouse-master  [.] memcpy
   4,35%  clickhouse-master  [.] _$LT$delta_kernel..engine..default..json..DefaultJsonHandler$LT$E$GT$$u20$as$u20$delta_kernel..JsonHandler$GT$::read_json_files::_$u7b$$u7b$clo
   2,56%  clickhouse-master  [.] arrow_json::reader::tape::TapeDecoder::finish
   2,28%  clickhouse-master  [.] std::sync::mpmc::Sender<T>::send
   2,14%  clickhouse-master  [.] <futures_util::stream::stream::map::Map<St,F> as futures_core::stream::Stream>::poll_next
   2,08%  clickhouse-master  [.] std::sync::mpmc::array::Channel<T>::start_send
   2,03%  clickhouse-master  [.] <futures_util::stream::try_stream::try_unfold::TryUnfold<T,F,Fut> as futures_core::stream::Stream>::poll_next
   1,89%  clickhouse-master  [.] <core::task::poll::Poll<core::option::Option<core::result::Result<T,F>>> as core::ops::try_trait::FromResidual<core::result::Result<core::conve
   1,89%  clickhouse-master  [.] delta_kernel::engine::default::json::JsonOpener::open::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::hce67c46c22d93036
   1,84%  clickhouse-master  [.] <futures_util::stream::try_stream::try_flatten::TryFlatten<St> as futures_core::stream::Stream>::poll_next
   1,70%  clickhouse-master  [.] <futures_util::stream::stream::map::Map<St,F> as futures_core::stream::Stream>::poll_next
   1,70%  clickhouse-master  [.] core::sync::atomic::AtomicUsize::load
   1,64%  clickhouse-master  [.] core::option::Option<T>::map
   1,57%  clickhouse-master  [.] alloc::fmt::format::format_inner
   1,44%  clickhouse-master  [.] std::sync::mpmc::array::Channel<T>::send
   1,39%  clickhouse-master  [.] std::sync::mpmc::Sender$LT$T$GT$::send::_$u7b$$u7b$closure$u7d$$u7d$::h89b262351b2aeb3c
   1,34%  clickhouse-master  [.] <core::pin::Pin<P> as futures_core::stream::Stream>::poll_next
   1,19%  clickhouse-master  [.] core::result::Result<core::option::Option<T>,E>::transpose
   1,15%  clickhouse-master  [.] <futures_util::stream::stream::map::Map<St,F> as futures_core::stream::Stream>::poll_next
   1,09%  clickhouse-master  [.] _$LT$futures_util..stream..stream..map..Map$LT$St$C$F$GT$$u20$as$u20$futures_core..stream..Stream$GT$::poll_next::_$u7b$$u7b$closure$u7d$$u7d$:
   1,07%  clickhouse-master  [.] <T as futures_util::fns::FnMut1<A>>::call_mut
   1,02%  clickhouse-master  [.] core::fmt::write
   0,99%  clickhouse-master  [.] core::option::Option<T>::as_pin_mut
   0,96%  clickhouse-master  [.] core::sync::atomic::AtomicBool::load
   0,93%  clickhouse-master  [.] core::ops::function::FnOnce::call_once
   0,92%  clickhouse-master  [.] core::option::Option<T>::map
   0,90%  clickhouse-master  [.] core::option::Option<T>::map
   0,86%  clickhouse-master  [.] <S as futures_core::stream::TryStream>::try_poll_next
   0,84%  clickhouse-master  [.] arrow_json::reader::Decoder::flush
   0,83%  clickhouse-master  [.] <S as futures_core::stream::TryStream>::try_poll_next
   0,82%  clickhouse-master  [.] <core::task::poll::Poll<core::option::Option<core::result::Result<T,E>>> as core::ops::try_trait::Try>::branch
   0,81%  clickhouse-master  [.] <futures_util::fns::MapErrFn<F> as futures_util::fns::FnMut1<core::result::Result<T,E>>>::call_mut
   0,80%  clickhouse-master  [.] core::result::Result<T,E>::map_err
   0,79%  clickhouse-master  [.] core::result::Result<T,E>::map_err
   0,76%  clickhouse-master  [.] <alloc::string::String as core::fmt::Write>::write_str
   0,76%  clickhouse-master  [.] <S as futures_core::stream::TryStream>::try_poll_next
   0,75%  clickhouse-master  [.] core::sync::atomic::atomic_load
   0,74%  clickhouse-master  [.] <core::pin::Pin<P> as futures_core::stream::Stream>::poll_next
   0,73%  clickhouse-master  [.] alloc::fmt::format
   0,70%  clickhouse-master  [.] core::slice::raw::from_raw_parts::precondition_check
   0,69%  clickhouse-master  [.] core::ptr::drop_in_place<std::sync::mpsc::SendError<core::result::Result<alloc::boxed::Box<dyn delta_kernel::engine_data::EngineData>,delta_ker
   0,69%  clickhouse-master  [.] <futures_util::stream::stream::next::Next<St> as core::future::future::Future>::poll
   0,65%  clickhouse-master  [.] alloc::fmt::format::_$u7b$$u7b$closure$u7d$$u7d$::h0511f2b35bbdd50a
   0,65%  clickhouse-master  [.] delta_kernel::error::Error::with_backtrace
   0,64%  clickhouse-master  [.] core::cmp::PartialOrd::le
   0,62%  clickhouse-master  [.] core::ptr::drop_in_place<std::backtrace::Backtrace>
   0,54%  clickhouse-master  [.] core::ptr::drop_in_place<arrow_schema::error::ArrowError>
   0,53%  clickhouse-master  [.] core::result::Result<T,E>::map
   0,49%  clickhouse-master  [.] core::ptr::drop_in_place<delta_kernel::error::Error>
   0,48%  clickhouse-master  [.] <core::result::Result<T,E> as core::ops::try_trait::Try>::branch
   0,48%  clickhouse-master  [.] core::ptr::drop_in_place<alloc::raw_vec::RawVec<u8>>
   0,47%  clickhouse-master  [.] <log::Level as core::cmp::PartialOrd<log::LevelFilter>>::partial_cmp
   0,47%  clickhouse-master  [.] std::sync::mpsc::SyncSender<T>::send
   0,46%  clickhouse-master  [.] <futures_util::stream::try_stream::MapErr<St,F> as futures_core::stream::Stream>::poll_next
   0,46%  clickhouse-master  [.] <futures_util::stream::try_stream::MapErr<St,F> as futures_core::stream::Stream>::poll_next
   0,45%  clickhouse-master  [.] <delta_kernel::error::Error as core::convert::From<arrow_schema::error::ArrowError>>::from
   0,45%  clickhouse-master  [.] <futures_util::stream::try_stream::into_stream::IntoStream<St> as futures_core::stream::Stream>::poll_next
   0,44%  clickhouse-master  [.] std::sync::mpmc::array::Channel<T>::write

Honestly I don't see much diff here. Yes it is visible, but it's kind of difficult to estimate the impact. Looks Ok for me.

@azat
Copy link
Copy Markdown
Member

azat commented Jul 25, 2025

Honestly I don't see much diff here

In a newer version with this sampling it shows that the overhead is 4.5%, what about the query execution times?

@nikitamikhaylov nikitamikhaylov force-pushed the everything-is-on-track branch from c0b661a to d3f3a5f Compare July 27, 2025 13:51
@nikitamikhaylov
Copy link
Copy Markdown
Member Author

nikitamikhaylov commented Jul 27, 2025

Stress test (msan): #83988

azat
azat reviewed Jul 28, 2025
View reviewed changes
@nikitamikhaylov nikitamikhaylov force-pushed the everything-is-on-track branch from cf60107 to bb1ede1 Compare July 29, 2025 16:36
@nikitamikhaylov
Copy link
Copy Markdown
Member Author

nikitamikhaylov commented Jul 30, 2025

Stateless tests (amd_asan, distributed plan): #84577

azat
azat reviewed Jul 30, 2025
View reviewed changes
Copy link
Copy Markdown
Member

@azat azat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apart from missing NULL handling in the *alloc functions, LGTM

@nikitamikhaylov
Copy link
Copy Markdown
Member Author

nikitamikhaylov commented Jul 31, 2025

Stateless tests (amd_asan, distributed plan, sequential), Stateless tests (amd_binary, ParallelReplicas, s3 storage, sequential): 00002_log_and_exception_messages_formatting #84537 #84538
Stateless tests (amd_binary, ParallelReplicas, s3 storage, parallel) #84771
Stateless tests (amd_tsan, s3 storage, parallel): 02443_detach_attach_partition #84173
Integration tests (tsan) :test_backup_restore_on_cluster/test.py::test_shutdown_waits_for_backup #84798

azat
azat reviewed Jul 31, 2025
View reviewed changes
@nikitamikhaylov nikitamikhaylov force-pushed the everything-is-on-track branch from 19dc7e4 to 378ece1 Compare August 6, 2025 11:29
@nikitamikhaylov
Copy link
Copy Markdown
Member Author

nikitamikhaylov commented Aug 6, 2025

Stateless tests (arm_binary, parallel) - interesting failure of 00601_kill_running_query.sh:
Basically the query executed before we started to wait for it:

2025.08.06 13:55:14.171407 [ 5269 ] {test_00601_test_x4viypd7} <Debug> HTTPServerRequest: Returning request input stream with ref count 1
2025.08.06 13:55:14.171413 [ 5269 ] {test_00601_test_x4viypd7} <Debug> HTTPServerRequest: creating in_post id 281302203247152
2025.08.06 13:55:14.171545 [ 5269 ] {test_00601_test_x4viypd7} <Debug> executeQuery: (from [::ffff:127.0.0.1]:58014) (comment: 00601_kill_running_query.sh) SELECT sum(ignore(*
)) FROM (SELECT number % 1000 AS k, groupArray(number) FROM numbers(50000000) GROUP BY k) SETTINGS max_rows_to_read = 0 (stage: Complete)
2025.08.06 13:55:14.171970 [ 5269 ] {test_00601_test_x4viypd7} <Trace> Planner: Query to stage Complete
2025.08.06 13:55:14.172012 [ 5269 ] {test_00601_test_x4viypd7} <Trace> Planner: Query to stage Complete
2025.08.06 13:55:14.172630 [ 5269 ] {test_00601_test_x4viypd7} <Trace> Planner: Query from stage FetchColumns to stage Complete
2025.08.06 13:55:14.172729 [ 5269 ] {test_00601_test_x4viypd7} <Trace> Planner: Query from stage FetchColumns to stage Complete
2025.08.06 13:55:14.174065 [ 358158 ] {test_00601_test_x4viypd7} <Trace> AggregatingTransform: Aggregating
2025.08.06 13:55:14.174292 [ 358158 ] {test_00601_test_x4viypd7} <Trace> Aggregator: Aggregation method: key16
2025.08.06 13:55:14.675810 [ 358158 ] {test_00601_test_x4viypd7} <Trace> AggregatingTransform: Aggregated. 50000000 to 1000 rows (from 476.84 MiB) in 0.502892752 sec. (9942477
7.552 rows/sec., 948.19 MiB/sec.)
2025.08.06 13:55:14.675825 [ 358158 ] {test_00601_test_x4viypd7} <Trace> Aggregator: Merging aggregated data
2025.08.06 13:55:14.675836 [ 358158 ] {test_00601_test_x4viypd7} <Trace> HashTablesStatistics: Statistics updated for key=97939551026471597: sum_of_sizes=1000, median_size=100
0
2025.08.06 13:55:14.756550 [ 358158 ] {test_00601_test_x4viypd7} <Debug> MemoryTracker: Query current memory usage: 1.42 GiB.
2025.08.06 13:55:14.864707 [ 358158 ] {test_00601_test_x4viypd7} <Trace> AggregatingTransform: Aggregating
2025.08.06 13:55:14.864724 [ 358158 ] {test_00601_test_x4viypd7} <Trace> Aggregator: Aggregation method: without_key
2025.08.06 13:55:14.864747 [ 358158 ] {test_00601_test_x4viypd7} <Trace> AggregatingTransform: Aggregated. 1000 to 1 rows (from 9.00 B) in 0.691807329 sec. (1445.489 rows/sec.
, 13.01 B/sec.)
2025.08.06 13:55:14.864757 [ 358158 ] {test_00601_test_x4viypd7} <Trace> Aggregator: Aggregation method: without_key
2025.08.06 13:55:14.864768 [ 358158 ] {test_00601_test_x4viypd7} <Trace> AggregatingTransform: Aggregated. 0 to 1 rows (from 0.00 B) in 0.691830984 sec. (0.000 rows/sec., 0.00
 B/sec.)
2025.08.06 13:55:14.864775 [ 358158 ] {test_00601_test_x4viypd7} <Trace> Aggregator: Merging aggregated data
2025.08.06 13:55:14.864781 [ 358158 ] {test_00601_test_x4viypd7} <Trace> HashTablesStatistics: Statistics updated for key=7584740618448132016: sum_of_sizes=2, median_size=1
2025.08.06 13:55:14.865159 [ 5269 ] {test_00601_test_x4viypd7} <Debug> executeQuery: Read 50000000 rows, 381.47 MiB in 0.693704 sec., 72076851.22184679 rows/sec., 549.90 MiB/s
ec.
2025.08.06 13:55:14.865213 [ 5269 ] {test_00601_test_x4viypd7} <Debug> DynamicQueryHandler: Done processing query
2025.08.06 13:55:14.865238 [ 5269 ] {test_00601_test_x4viypd7} <Debug> MemoryTracker: Query peak memory usage: 1.42 GiB.
2025.08.06 13:55:15.483066 [ 12342 ] {54ca4524-9868-43fd-97ef-c4b6b0336192} <Debug> executeQuery: (from [::ffff:127.0.0.1]:58088) (comment: 00601_kill_running_query.sh) SELECT
 count() FROM system.processes WHERE query_id = 'test_00601_test_x4viypd7' (stage: Complete)
2025.08.06 13:55:16.968705 [ 12083 ] {6147aa68-990b-49f2-b85b-6068fe0bc427} <Debug> executeQuery: (from [::ffff:127.0.0.1]:58134) (comment: 00601_kill_running_query.sh) SELECT
 count() FROM system.processes WHERE query_id = 'test_00601_test_x4viypd7' (stage: Complete)
2025.08.06 13:55:17.555438 [ 313530 ] {eb3c8b7e-1624-4618-b539-91eaed0efb78} <Debug> executeQuery: (from [::ffff:127.0.0.1]:34058) (comment: 00601_kill_running_query.sh) SELEC
T count() FROM system.processes WHERE query_id = 'test_00601_test_x4viypd7' (stage: Complete)
2025.08.06 13:55:17.702266 [ 12342 ] {3d522ad3-7ba6-464e-a42c-24feb772f3b3} <Debug> executeQuery: (from [::ffff:127.0.0.1]:34188) (comment: 00601_kill_running_query.sh) SELECT
 count() FROM system.processes WHERE query_id = 'test_00601_test_x4viypd7' (stage: Complete)
2025.08.06 13:55:17.826241 [ 5470 ] {dba76501-f994-402c-b641-713c8b159140} <Debug> executeQuery: (from [::ffff:127.0.0.1]:34276) (comment: 00601_kill_running_query.sh) SELECT
count() FROM system.processes WHERE query_id = 'test_00601_test_x4viypd7' (stage: Complete)
2025.08.06 13:55:17.946553 [ 40473 ] {62f117c8-da17-4879-ad1a-b90529b31e6e} <Debug> executeQuery: (from [::ffff:127.0.0.1]:34372) (comment: 00601_kill_running_query.sh) SELECT
 count() FROM system.processes WHERE query_id = 'test_00601_test_x4viypd7' (stage: Complete)
2025.08.06 13:55:18.082423 [ 313572 ] {36c6f5ad-471f-4737-8ec8-7714db70087d} <Debug> executeQuery: (from [::ffff:127.0.0.1]:34452) (comment: 00601_kill_running_query.sh) SELEC
T count() FROM system.processes WHERE query_id = 'test_00601_test_x4viypd7' (stage: Complete)

Stress tests - Logical error: \'Expected JOIN table expression to be table, table function, query or union node. Actual ANY LEFT JOIN ... ON k1 = mem.k\' #63984

@nikitamikhaylov nikitamikhaylov added this pull request to the merge queue Aug 6, 2025
Merged via the queue into master with commit 7ea6a89 Aug 6, 2025
121 of 124 checks passed
@nikitamikhaylov nikitamikhaylov deleted the everything-is-on-track branch August 6, 2025 17:08
@robot-ch-test-poll3 robot-ch-test-poll3 added the pr-synced-to-cloud The PR is synced to the cloud repo label Aug 6, 2025
baibaichen pushed a commit to Kyligence/gluten that referenced this pull request Aug 7, 2025
Fix build due to ClickHouse/ClickHouse#84082
cc9fb90
@Algunenano Algunenano mentioned this pull request Aug 7, 2025
Closed
baibaichen pushed a commit to Kyligence/gluten that referenced this pull request Aug 8, 2025
Fix build due to ClickHouse/ClickHouse#84082
7c738bf
baibaichen pushed a commit to Kyligence/gluten that referenced this pull request Aug 11, 2025
Fix build due to ClickHouse/ClickHouse#84082
e2b1ac4
baibaichen pushed a commit to Kyligence/gluten that referenced this pull request Aug 12, 2025
Fix build due to ClickHouse/ClickHouse#84082
714985a
baibaichen pushed a commit to Kyligence/gluten that referenced this pull request Aug 13, 2025
Fix build due to ClickHouse/ClickHouse#84082
8e49c7f
baibaichen pushed a commit to Kyligence/gluten that referenced this pull request Aug 14, 2025
Fix build due to ClickHouse/ClickHouse#84082
e792a57
baibaichen pushed a commit to Kyligence/gluten that referenced this pull request Aug 15, 2025
Fix build due to ClickHouse/ClickHouse#84082
d6b955e
baibaichen pushed a commit to Kyligence/gluten that referenced this pull request Aug 16, 2025
Fix build due to ClickHouse/ClickHouse#84082
bcf06b2
baibaichen pushed a commit to Kyligence/gluten that referenced this pull request Aug 18, 2025
Fix build due to ClickHouse/ClickHouse#84082
7c55d2b
@den-crane den-crane mentioned this pull request Sep 2, 2025
Merged
1 task
Avogar added a commit that referenced this pull request Sep 2, 2025
@Avogar
Move #84082 to the correct section
8855cc4
@nikitamikhaylov nikitamikhaylov mentioned this pull request Oct 2, 2025
Merged
1 task
@azat azat mentioned this pull request Nov 15, 2025
Merged
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@alexey-milovidov alexey-milovidov alexey-milovidov left review comments

@azat azat azat approved these changes

Assignees

@azat azat

Labels

pr-improvement Pull request with some product improvements pr-synced-to-cloud The PR is synced to the cloud repo submodule changed At least one submodule changed in this PR.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@nikitamikhaylov @azat @alexey-milovidov @robot-ch-test-poll3