Skip to content

mallctl("epoch") is very slow in some scenario #1502

@oranagra

Description

@oranagra

Hi, I'm trying to enable the background_thread in Redis to solve a problem i reported in #1398.
while testing for possible regression i found a problem which i'm not yet sure what it is related to.

I'm attaching a small tester that reproduces it, but the general scenario is this:
Application allocates some massive amount of allocations (about 100M allocations of 100 bytes, 100M allocations of 24 bytes, and another 8MB of big chunk).
It then proceeds to release that memory in a background thread, while the main thread starts working on a new data set (but in some cases the main thread may also go idle).
Since the main thread may go idle, i'm enabling the background_thread.
Another thing that the main thread is doing from time to time is collect some stats from jemalloc, for which it needs to set the epoch using `mallctl.

In some cases, i observed that triggering the epoch calculations takes over a second!!!
This doesn't seem to occur when the background_thread is disabled, and it also doesn't seem to occur if the old allocations are released at the same order in which they were allocated (this is why the attached tester uses Redis's dict.c, i.e. to scramble the order in which deallocations are make)

Please let me know if there's a way to mitigate this? or if this is maybe a bug / edge case that can be fixed in jemalloc.

Thanks.

test_epoch_calc_time.tgz.txt
(sorry for the dirty coding)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions