Provide configurable memory limit for Conn tracker buffer usage#2196
Merged
ddelnano merged 1 commit intopixie-io:mainfrom May 28, 2025
Merged
Conversation
Signed-off-by: Dom Del Nano <[email protected]> (cherry picked from commit aea0a95)
oazizi000
approved these changes
May 28, 2025
ddelnano
added a commit
to k8sstormcenter/pixie
that referenced
this pull request
Feb 25, 2026
…e-io#2196) Summary: Provide configurable memory limit for Conn tracker buffer usage Some memory conscious users want to prevent PEM's from having unbounded memory use. The socket tracer's conn tracker component allows unbounded memory growth due to storing an uncapped amount of socket data. From profiling the [PEM's heap use](https://github.com/pixie-io/pixie/blob/301198f2af6739f0de8b01e1cbcb3451b3075e28/scripts/collect_heap_pprofs.sh), this memory growth is a significant contributor to the PEM's memory footprint. By leveraging the existing [table store limit](https://github.com/pixie-io/pixie/blob/301198f2af6739f0de8b01e1cbcb3451b3075e28/src/table_store/table/table.cc#L44) and this new configuration option, these memory conscious users run Pixie with a smaller and more predictable steady state memory usage. Relevant Issues: N/A Type of change: /kind feature Test Plan: Ran perf_tool and k9s based load tests to verify the following: - [x] Verified results by running PEM's with a 0.5-1 GiB memory limit - [x] Verified Conn tracker buffers no longer showed up as significant memory contributors in heap profiles. Changelog Message: PEM's can now limit the total memory used by Pixie's socket tracer buffers through the `--total_conn_tracker_mem_usage` command line flag or the `PX_TOTAL_CONN_TRACKER_MEM_USAGE` env var. This allows memory conscious users to have more predictable memory use when paired with `PL_TABLE_STORE_TABLE_SIZE_LIMIT`. Signed-off-by: Dom Del Nano <[email protected]>
ddelnano
added a commit
to k8sstormcenter/pixie
that referenced
this pull request
Feb 25, 2026
…e-io#2196) Summary: Provide configurable memory limit for Conn tracker buffer usage Some memory conscious users want to prevent PEM's from having unbounded memory use. The socket tracer's conn tracker component allows unbounded memory growth due to storing an uncapped amount of socket data. From profiling the [PEM's heap use](https://github.com/pixie-io/pixie/blob/301198f2af6739f0de8b01e1cbcb3451b3075e28/scripts/collect_heap_pprofs.sh), this memory growth is a significant contributor to the PEM's memory footprint. By leveraging the existing [table store limit](https://github.com/pixie-io/pixie/blob/301198f2af6739f0de8b01e1cbcb3451b3075e28/src/table_store/table/table.cc#L44) and this new configuration option, these memory conscious users run Pixie with a smaller and more predictable steady state memory usage. Relevant Issues: N/A Type of change: /kind feature Test Plan: Ran perf_tool and k9s based load tests to verify the following: - [x] Verified results by running PEM's with a 0.5-1 GiB memory limit - [x] Verified Conn tracker buffers no longer showed up as significant memory contributors in heap profiles. Changelog Message: PEM's can now limit the total memory used by Pixie's socket tracer buffers through the `--total_conn_tracker_mem_usage` command line flag or the `PX_TOTAL_CONN_TRACKER_MEM_USAGE` env var. This allows memory conscious users to have more predictable memory use when paired with `PL_TABLE_STORE_TABLE_SIZE_LIMIT`. Signed-off-by: Dom Del Nano <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary: Provide configurable memory limit for Conn tracker buffer usage
Some memory conscious users want to prevent PEM's from having unbounded memory use. The socket tracer's conn tracker component allows unbounded memory growth due to storing an uncapped amount of socket data. From profiling the PEM's heap use, this memory growth is a significant contributor to the PEM's memory footprint.
By leveraging the existing table store limit and this new configuration option, these memory conscious users run Pixie with a smaller and more predictable steady state memory usage.
Relevant Issues: N/A
Type of change: /kind feature
Test Plan: Ran perf_tool and k9s based load tests to verify the following:
Changelog Message: PEM's can now limit the total memory used by Pixie's socket tracer buffers through the
--total_conn_tracker_mem_usagecommand line flag or thePX_TOTAL_CONN_TRACKER_MEM_USAGEenv var. This allows memory conscious users to have more predictable memory use when paired withPL_TABLE_STORE_TABLE_SIZE_LIMIT.