Skip to content

[low-priority] Close bridged instruments when GC'd#16219

Merged
laurit merged 9 commits intoopen-telemetry:mainfrom
trask:close-bridged-instruments
Mar 30, 2026
Merged

[low-priority] Close bridged instruments when GC'd#16219
laurit merged 9 commits intoopen-telemetry:mainfrom
trask:close-bridged-instruments

Conversation

@trask
Copy link
Copy Markdown
Member

@trask trask commented Feb 18, 2026

Currently, when the agent bridges an async instrument callback (e.g. buildWithCallback), the SDK
ends up holding a strong reference chain back to the app classloader:

Agent SDK → bridging lambda → user callback → callback class → app classloader

This pins the app classloader forever, leaking memory on tomcat webapp redeploys.

This PR breaks the chain with a WeakReference, then anchors the bridging lambda to the app classloader's own lifecycle so it stays alive exactly as long as the classloader does:

Agent SDK → WeakRefConsumer (bootstrap CL, no leak)
                ↓ weak ref
            bridging lambda (app CL)
                ↑ strong ref (anchoring)
            CallbackAnchor.callbacks (app CL)
                ↑ static field
            CallbackAnchor.class (app CL)
                ↑ class
            app classloader

(all good ideas here are credit to @laurit, all bad implementation details are credit to me)

@trask trask force-pushed the close-bridged-instruments branch from 65b1b94 to 72ebbb8 Compare February 18, 2026 04:49
@trask trask marked this pull request as ready for review February 18, 2026 06:00
@trask trask requested a review from a team as a code owner February 18, 2026 06:00
@trask trask changed the title Close bridged instruments when GC'd [low-priority] Close bridged instruments when GC'd Feb 27, 2026
Consumer<application.io.opentelemetry.api.metrics.ObservableDoubleMeasurement>
applicationCallback) {
return new ApplicationObservableDoubleCounter(
agentBuilder.buildWithCallback(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we also handle calling close on the returned ObservableDoubleCounter and remove the callback from the list so it could be collected immediately?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@trask trask force-pushed the close-bridged-instruments branch from 817261c to 4c03a99 Compare March 10, 2026 04:18

// Anchors callbacks to this class's lifecycle. Since this class is injected as a helper into each
// application class loader, callbacks are naturally tied to their class loader's lifecycle.
private static final Set<Object> callbacks = ConcurrentHashMap.newKeySet();
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally I think this should be an identity set to guard agains callbacks with weird equals/hashCode. To handle using the same callback for multiple instruments where one of the instruments gets closed it would need to use ref counting. I think we can ignore these for now.

@laurit laurit merged commit 6098ba7 into open-telemetry:main Mar 30, 2026
93 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants