Add simple memory profiler#8765
Conversation
abyss7
force-pushed
the
memory-profiler
branch
from
f06df75 to
8b8300d
Compare
abyss7
force-pushed
the
memory-profiler
branch
from
8b8300d to
d444bc7
Compare
abyss7
force-pushed
the
memory-profiler
branch
from
976adce to
2899fbf
Compare
alexey-milovidov
left a comment
alexey-milovidov
left a comment
There was a problem hiding this comment.
I have some questions in TG chat.
|
Test failure is related. |
Yes, I will fix it. |
dbms/src/Interpreters/TraceLog.cpp
Outdated
| {std::make_shared<DataTypeString>(), "query_id"}, | ||
| {std::make_shared<DataTypeArray>(std::make_shared<DataTypeUInt64>()), "trace"} | ||
| {std::make_shared<DataTypeArray>(std::make_shared<DataTypeUInt64>()), "trace"}, | ||
| {std::make_shared<DataTypeInt64>(), "size"}, |
There was a problem hiding this comment.
I'd suggest allocated_bytes.
There was a problem hiding this comment.
It's Ok. Because what else we will add to the trace_log.
|
It is not possible to initialize singleton inside signal handler: |
| sizeof(UInt32) + // thread_number | ||
| sizeof(Int64); // size | ||
| char buffer[buf_size]; | ||
| WriteBufferFromFileDescriptorDiscardOnFailure out(pipe.fds_rw[1], buf_size, buffer); |
There was a problem hiding this comment.
Is it time to write a ser/de method for TraceLogElement? Or to even switch a trace from Array to std::array, so that TraceLogElement is fixed size and we can ship it verbatim over the pipe. Actually we can ship it even if it has pointers inside, because we're in the same process -- this would have move semantics. But I don't trust these pipes enough, maybe it'll leak :)
There was a problem hiding this comment.
switch a trace from Array to std::array, so that TraceLogElement is fixed size
No need for that. When traces comes frequently, pipe is overflowed.
It allows to workaround first call inside signal handler
It's because the |
|
BTW, non-trivial static function variables contain spinlock and mutex - cannot be used in signal handlers. |
|
You forgot to mention about backward incompatible changes in |
5 participants
I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en
Changelog category (leave one):
Changelog entry (up to few sentences, required except for Non-significant/Documentation categories):
Implement simple memory-profiler that dumps stacktraces to system.trace_log every N bytes over soft allocation limit