Skip to content

Log msg memory accounting fixes#889

Merged
sodomelle merged 9 commits intoaxoflow:mainfrom
bazsi:log-msg-memory-accounting-fixes
Jan 29, 2026
Merged

Log msg memory accounting fixes#889
sodomelle merged 9 commits intoaxoflow:mainfrom
bazsi:log-msg-memory-accounting-fixes

Conversation

@bazsi
Copy link
Member

@bazsi bazsi commented Dec 20, 2025

The calculation behind the syslogng_memory_queue_memory_usage_bytes was incorrect in various ways:

  • the NVTable uses the entire table, and not just the memory we suballocated
  • the LogMessage structure did not account for the nodes array

This PR fixes both of these problems.

@bazsi bazsi force-pushed the log-msg-memory-accounting-fixes branch 3 times, most recently from cbd6bf4 to ad64ac4 Compare December 23, 2025 20:48
@bazsi bazsi force-pushed the log-msg-memory-accounting-fixes branch 2 times, most recently from 5b0328a to eae92a8 Compare December 29, 2025 11:23
@bazsi bazsi force-pushed the log-msg-memory-accounting-fixes branch from eae92a8 to 432096f Compare January 24, 2026 13:45
Copy link
Contributor

@sodomelle sodomelle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I marked a few things for fix/clarification.

@bazsi bazsi force-pushed the log-msg-memory-accounting-fixes branch from 432096f to ebc657a Compare January 28, 2026 09:06
@bazsi
Copy link
Member Author

bazsi commented Jan 28, 2026

@sodomelle I am following up light changes so the CI becomes green. I am also thinking about fixing the memory use for the "backlog", right now they are not part of the memory usage, but I think they should be.

@bazsi bazsi force-pushed the log-msg-memory-accounting-fixes branch 2 times, most recently from 67fc2f8 to d45b3c8 Compare January 28, 2026 21:45
bazsi added 9 commits January 28, 2026 22:54
We are actually allocating a larger chunk of memory, not just as much
as we actually use.

Signed-off-by: Balazs Scheidler <[email protected]>
Cover msg->tags and the socket addresses.

Signed-off-by: Balazs Scheidler <[email protected]>
This patch removes the test the goes into implementation details about how
we allocate the LogMessage.  This is really fragile and does not add much
value, it's best to remove it.

Signed-off-by: Balazs Scheidler <[email protected]>
Also, make it inline.

Signed-off-by: Balazs Scheidler <[email protected]>
… enough value

These tests implicitly test the LogMessage memory allocation, which can
be affected by the tests themselves, thus the memory allocation
of a single LogMessage was not constant. With this value change, this can
assumed to not change.

Signed-off-by: Balazs Scheidler <[email protected]>
Memory usage is not stable between reload/restart operations, so we can't
really test them. The other testcases have these disabled already, so
follow suit.

Signed-off-by: Balazs Scheidler <[email protected]>
Signed-off-by: Balazs Scheidler <[email protected]>
@bazsi bazsi force-pushed the log-msg-memory-accounting-fixes branch from d45b3c8 to b5a707c Compare January 29, 2026 08:28
@bazsi
Copy link
Member Author

bazsi commented Jan 29, 2026

@sodomelle I addressed the review comments, added a news file and made the CI green, so I think this should be good to go in, can you recheck this and add your approval? Thank you.

@sodomelle sodomelle merged commit 44534ce into axoflow:main Jan 29, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants