Skip to content

Add live allocation diagnostics#447

Merged
nsavoire merged 1 commit into
mainfrom
nsavoire/add_live_alloc_diagnostics
Oct 13, 2025
Merged

Add live allocation diagnostics#447
nsavoire merged 1 commit into
mainfrom
nsavoire/add_live_alloc_diagnostics

Conversation

@nsavoire

@nsavoire nsavoire commented Oct 13, 2025

Copy link
Copy Markdown
Collaborator

Description

This PR enhances the live allocation profiling system with comprehensive diagnostic capabilities to better track and troubleshoot allocation/deallocation mismatches and data loss.

Key Changes:

1. New Performance Event Types

  • Added PERF_CUSTOM_EVENT_LOST_ALLOCATION to track lost allocation and deallocation events separately
  • Added PERF_CUSTOM_EVENT_ALLOCATION_TRACKER_STATE to periodically report the allocation tracker's internal state

2. Enhanced Event Structures

  • LostAllocationEvent: Replaces generic lost events with separate counters for lost_alloc_count and lost_dealloc_count
  • AllocationTrackerStateEvent: Reports tracked_addresse_count and address_conflict_count from the library

3. Improved Statistics Tracking

  • Added EVENT_DEALLOC_LOST stat to separately track lost deallocation events
  • Added ALREADY_EXISTING_ALLOCATION_COUNT stat to count duplicate allocations at the same address (indicating missed free events)

4. Allocation Tracker Enhancements

  • Split lost_count into separate lost_alloc_count and lost_dealloc_count atomics for granular tracking
  • Added address_conflict_count to track hash collisions in the address set
  • Implemented push_allocation_tracker_state() to periodically report library state to the profiler
  • Integrated state reporting with the timer callback mechanism

5. Live Allocation Diagnostics

  • Added register_library_state() method to synchronize library state with profiler's view
  • Tracks address_conflict_count and tracked_address_count per PID

6. Enhanced Logging and Visibility

  • Consolidated diagnostic logging in register_library_state() showing:
    • Live allocations tracked by profiler (_address_map.size())
    • Unique stacks (_unique_stacks.size())
    • Library-tracked addresses (_tracked_address_count)
    • Address conflicts from library (_address_conflict_count)
  • Above log was moved from aggregation phase, because consistency between library and profiler can only be assessed when library sends its status
  • Added debug logging for clearing live allocations and state reporting

Benefits:

These diagnostics provide better visibility into allocation tracking health, making it easier to identify when events are lost, when allocations/deallocations are mismatched, and when the library and profiler states diverge. This is crucial for debugging live heap profiling in production environments.

@nsavoire
nsavoire requested a review from r1viollet as a code owner October 13, 2025 09:32
@pr-commenter

pr-commenter Bot commented Oct 13, 2025

Copy link
Copy Markdown

Benchmark results for collatz

Parameters

Baseline Candidate
config baseline candidate
profiler-version ddprof 0.20.0+a69c81bc.78847135 ddprof 0.20.0+03acfa5d.79131052

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 1 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean execution_time
scenario:ddprof -S bench-collatz --preset cpu_only collatz_runner.sh same

@pr-commenter

pr-commenter Bot commented Oct 13, 2025

Copy link
Copy Markdown

Benchmark results for BadBoggleSolver_run

Parameters

Baseline Candidate
config baseline candidate
profiler-version ddprof 0.20.0+a69c81bc.78847135 ddprof 0.20.0+03acfa5d.79131052

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 1 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean execution_time
scenario:ddprof -S bench-bad-boggle-solver BadBoggleSolver_run work 1000 unsure
[+0.038ms; +7.667ms] or [+0.002%; +0.389%]

@nsavoire
nsavoire force-pushed the nsavoire/add_live_alloc_diagnostics branch from 13ceda7 to 9e75ba3 Compare October 13, 2025 10:54
r1viollet
r1viollet previously approved these changes Oct 13, 2025

@r1viollet r1viollet left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@nsavoire
nsavoire force-pushed the nsavoire/add_live_alloc_diagnostics branch from 9e75ba3 to e166a13 Compare October 13, 2025 11:38
@nsavoire
nsavoire requested a review from r1viollet October 13, 2025 11:41
@nsavoire
nsavoire force-pushed the nsavoire/add_live_alloc_diagnostics branch from e166a13 to 03acfa5 Compare October 13, 2025 11:42

@r1viollet r1viollet left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@nsavoire
nsavoire merged commit ea712fb into main Oct 13, 2025
2 checks passed
@nsavoire
nsavoire deleted the nsavoire/add_live_alloc_diagnostics branch October 13, 2025 12:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants