Skip to content

Merge feature/continuous-profiling into develop#3622

Merged
ambushwork merged 113 commits into
developfrom
yl/merge-develop-into-feature-profiling
Jul 9, 2026
Merged

Merge feature/continuous-profiling into develop#3622
ambushwork merged 113 commits into
developfrom
yl/merge-develop-into-feature-profiling

Conversation

@ambushwork

Copy link
Copy Markdown
Member

What does this PR do?

Merge develop into feature/continuous-profiling

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Make sure you discussed the feature or bugfix with the maintaining team in an Issue
  • Make sure each commit and the PR mention the Issue number (cf the CONTRIBUTING doc)

ambushwork and others added 30 commits March 20, 2026 11:55
…us-sample-rate-api

RUM-15189: Add setContinuousSampleRate API to ProfilingConfiguration
…-continuous-profiling-250326

Merge `develop` branch into `feature/continuous-profiling` branch
…aunch-duration

RUM-15190: Extend app launch profiling duration for continuous merge
…event-to-profiling

Send event to Profiling feature when RUM session is renewed
…us-profiling-scheduler

RUM-15191: Implement ContinuousProfilingScheduler
…inuous-profiling

RUM-15323: RUM sends ANR & LongTask events to Profiling Feature
…lback

RUM-15307: Report failure through ProfilerCallback
…e-profiling-rum-context

RUM-15334: Commonize ProfilingRumContext for TTID and Continuous Profiling
…ing-rum-event

RUM-15334: Add id to RumAnrEvent and RumLongTaskEvent
…filing-cycle

Fix continuous profiling back-to-back with app launch by inserting a cooldown window
…-to-scheduler

RUM-15321: Continuous profiling writes are gated on pending RUM events
…-continuous-profiling-030426

Merge `develop` branch into `feature/continuous-profiling`
…ndicator

RUM-15454: Add Profiler status in RUM debug widget
@ambushwork
ambushwork requested review from a team as code owners July 9, 2026 07:01

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0407de65dd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines 139 to +142
sdkCore.getFeature(Feature.PROFILING_FEATURE_NAME)?.sendEvent(
ProfilerStopEvent.TTID(
rumContext = TTIDRumContext(
// startMs is different for different scenarios. For Cold it will be process start, for Warm
// it will be Activity.onCreate. But we have application launch profiling working for cold starts only anyway.
ProfilerEvent.RumVitalEvent(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Validate TTID before publishing it to profiling

For TTID values over MAX_TTID_DURATION_NS, sendTTIDEvent later drops the RUM vital, but this profiling event has already been queued with the vital id. In that scenario the uploaded profile can reference a TTID vital that was never written to RUM, breaking profile/RUM correlation; perform the same duration validation before sending the profiler event, or only send it after the RUM event is accepted for writing.

Useful? React with 👍 / 👎.

override fun onAnrDetected(event: ProfilingAnrDetectedEvent) {
// The ANR event should be forwarded to RUM only when profiling is actually running.
if (isLaunchProfilingActive || continuousProfilingScheduler?.isActive == true) {
sdkCore.getFeature(Feature.RUM_FEATURE_NAME)?.sendEvent(event)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Honor RUM's non-fatal ANR opt-out

On API 30+ trackNonFatalAnrs is disabled by default (and users can explicitly disable it), but this forwarding path sends every profiling-trigger ANR to the RUM feature whenever a profile is active. RumFeature handles ProfilingAnrDetectedEvent by calling addError unconditionally, bypassing the existing configuration.trackNonFatalAnrs guard used to start the watchdog detector, so enabling profiling can report non-fatal ANR errors for apps that opted out. Gate this event on the RUM ANR setting or carry the setting into this path before sending.

Useful? React with 👍 / 👎.

Comment on lines +296 to +303
if (quotaResult.decision == QuotaResult.Decision.DENIED) {
logToUser(
LOG_LAUNCH_PROFILING_DROPPED_QUOTA_DENIED.format(
Locale.US,
quotaResult.reason.rawValue
)
)
pendingRumEvents.clear()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Delete traces dropped by quota

When the launch profile is denied by quota, this branch consumes perfettoResult and never passes result.resultFilePath to ProfilingDataWriter, which is currently the only path that deletes the Perfetto file. Every quota-denied launch profile therefore leaves its trace file behind even though it will never be uploaded; delete the result file in the denied branch before clearing the buffered state.

Useful? React with 👍 / 👎.

Comment on lines +261 to +263
val sampleRate = (context[FeatureContextKeys.RUM_SESSION_SAMPLE_RATE] as? Number)?.toFloat()
?: DEFAULT_RUM_SESSION_SAMPLE_RATE
lastSeenRumSessionId = sessionId

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Skip sessions RUM did not track

This renewal path samples profiling from the RUM session id and sample rate but ignores the RUM session_state that is also present in the feature context. For a RUM session that sampled out (NOT_TRACKED) while the configured sample rate is still non-zero, profiling can mark the session as sampled, extend the launch profile, and start continuous windows even though RUM is writing through a no-op writer, producing profiles that either have no usable RUM events or reference events RUM never ingested. Check that the RUM session is actually tracked before running quota checks or renewing the profiling scheduler.

Useful? React with 👍 / 👎.

kikoveiga
kikoveiga previously approved these changes Jul 9, 2026
@ambushwork
ambushwork force-pushed the yl/merge-develop-into-feature-profiling branch from 0407de6 to 1371e7d Compare July 9, 2026 08:10

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1371e7d579

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

runnable = {
val cooldownMs = jitter(CONTINUOUS_COOLDOWN_DURATION_MS)
logToUser { LOG_ACTIVE_WINDOW_ENDED.format(Locale.US, cooldownMs) }
scheduleCooldown(cooldownMs)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Stop buffering when the active window expires

When a continuous window reaches this scheduled end, the scheduler moves into cooldown but leaves isActive true until the Perfetto result callback later calls onActiveWindowEnded(). If that callback is delayed, ProfilingFeature.isRecordingProfile() continues accepting long tasks, ANRs, and vitals that occurred after the trace interval, so the completed profile can be uploaded with out-of-window RUM metadata; clear the active flag here before entering cooldown, or gate buffering on the actual profiler state.

Useful? React with 👍 / 👎.

eventType = EventType.DEFAULT
)
}
safeDelete(profilingResult.resultFilePath)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Defer deleting shared traces until all instances read them

When launch profiling was started for multiple SDK instances, PerfettoProfiler fans the same resultFilePath out to every registered callback, but the first ProfilingDataWriter that runs deletes that shared file here. The remaining instance callbacks then try to read a missing file and drop their profile, so only one SDK instance can upload the trace; delete the file only after all callbacks have copied it, or share the bytes before any writer deletes the path.

Useful? React with 👍 / 👎.

val anrDurationNs = TimeUnit.MILLISECONDS.toNanos(anrDurationMs)

profilingStatus = resolveErrorProfilingStatus(datadogContext)
sdkCore.getFeature(Feature.PROFILING_FEATURE_NAME)?.sendEvent(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Publish ANR metadata after the error is written

For a non-fatal ANR, this publishes the profiling metadata before the RUM ErrorEvent write operation runs. If the app's errorEventMapper drops the non-crash error or the write fails, the uploaded profile can still reference this error id even though RUM never ingested it; move the profiling send into the write operation's success path using the captured context and id.

Useful? React with 👍 / 👎.

)

val longTaskId = UUID.randomUUID().toString()
sdkCore.getFeature(Feature.PROFILING_FEATURE_NAME)?.sendEvent(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Publish long-task metadata after successful writes

This sends the long-task id to profiling before the corresponding RUM LongTaskEvent is actually accepted by the writer. When a longTaskEventMapper drops the event or storage write fails, the profile can still be uploaded with a long-task id that does not exist in RUM; use the write operation's success callback, as the operation-vital path does, before buffering the profiling metadata.

Useful? React with 👍 / 👎.

}
tagsProfiler = buildTags(context, profilingResult.startReason.value),
application = ProfileEvent.Application(id = rumContext.applicationId),
session = ProfileEvent.Session(id = rumContext.sessionId),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Split profile metadata by RUM session

A continuous window can overlap a RUM session renewal or explicit session reset, but the pending buffer is not drained on renewal and this writes a single profile session.id from the first buffered event while still adding all long-task/error/vital ids from the whole buffer. In that scenario the uploaded profile references events from the later session under the earlier session id, so correlation for those events is wrong; split or drop buffered metadata when the RUM session changes.

Useful? React with 👍 / 👎.

private fun onTtidEvent() {
if (isTtidVitalReceived.getAndSet(true)) return

if (continuousProfilingScheduler?.currentSessionSampled != true) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Stop launch traces once quota denies them

When the first RUM session is sampled for continuous profiling, this TTID path does not stop the launch trace, even if the quota result is already denied or becomes denied just after TTID. The trace then keeps running for the extended launch window only to be dropped in tryWriteProfilingEvent, which wastes profiling overhead for users whose org/session is quota-denied; include the quota decision in this stop condition or stop when a denial arrives.

Useful? React with 👍 / 👎.

}
)

sdkCore.getFeature(Feature.PROFILING_FEATURE_NAME)?.sendEvent(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Publish TTFD metadata after successful writes

This sends the TTFD vital id to profiling before the VitalAppLaunchEvent write is submitted. If a vitalAppLaunchEventMapper drops the TTFD event or storage rejects the write, the buffered profile can still reference a vital id that RUM never ingested; mirror the operation-vital path and send this profiling event from the write operation's success callback.

Useful? React with 👍 / 👎.

scheduleCooldown(activeMs)
return
}
onActiveWindowStarted()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Wait for the previous result before clearing events

If a Perfetto result callback is delayed past the cooldown, scheduleNextCycle() can run while PerfettoProfiler still considers the previous request active, so the next profiler.start no-ops but onActiveWindowStarted() has already cleared pendingRumEvents. That loses the RUM metadata needed by the previous profile and marks a new active window without an actual trace; don't clear/start a new window until the prior result callback has completed or the profiler confirms a new request started.

Useful? React with 👍 / 👎.

@ambushwork
ambushwork force-pushed the yl/merge-develop-into-feature-profiling branch from 1371e7d to 54580e9 Compare July 9, 2026 08:41

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 54580e9ceb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

)
}.onSuccess {
sendProfilingRumOperationEvent(
rumContext = capturedRumContext,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Use the written RUM context for operation vitals

When writeScope is delayed long enough for a session/view renewal, newVitalEvent() builds the RUM vital from the later getRumContext() on the storage worker, but this callback publishes the profiler metadata with the earlier capturedRumContext. In that scenario the operation vital is written under the new RUM session/view while the profile buffers the same vital id under the old session/view, breaking correlation; publish from the event that was actually written or build both from the same captured context.

Useful? React with 👍 / 👎.

// fatal-ANR replay path (DatadogLateCrashReporter, via ApplicationExitInfo on
// the next launch) can dedupe against an ANR the in-process watchdog/profiling
// trigger already reported. The storage key keeps the legacy "fatal" name.
sdkCore.writeLastFatalAnrSent(event.eventTime.timestamp)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid deduping distinct fatal ANRs

Persisting every successful non-fatal ANRException into the fatal-ANR marker makes DatadogLateCrashReporter skip any later ApplicationExitInfo ANR whose timestamp is within the new ±10s tolerance. If an app recovers from one ANR and then is killed by a separate ANR within that window (or restarts quickly and hangs again), the fatal crash is treated as already sent and is never reported; store a marker that identifies the replayed ANR source more precisely instead of using this shared timestamp for all non-fatal ANRs.

Useful? React with 👍 / 👎.

val isRunning = datadogContext.isProfilerRunning()
val quotaReason = datadogContext.resolveProfilingQuotaReason(sessionId)
return when {
isRunning -> ViewEvent.Profiling(status = ViewEvent.ProfilingStatus.RUNNING)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include clock drift on running view events

The shared profiling schema says clock_drift is present when the profiler is active, and the error/long-task/vital paths populate it, but view updates marked RUNNING omit it here. For sessions with a non-zero NTP offset, consumers using the view event's profiling context do not get the offset needed to align RUM timestamps with the profile; pass datadogContext.time.serverTimeOffsetMs as clockDrift, as in the other RUM event paths.

Useful? React with 👍 / 👎.

Comment on lines +264 to +265
sdkCore.getFeature(Feature.PROFILING_FEATURE_NAME)?.withContext { datadogContext ->
quotaChecker.checkAsync(sessionId, datadogContext)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Skip quota checks for sampled-out profiling sessions

This quota request runs on every RUM session renewal before knowing whether profiling will actually run. When continuous profiling is disabled or the profiling sampler excludes the session and no launch trace is active, a DENIED quota response still publishes PROFILING_QUOTA_REASON, so later RUM events are tagged as profiling STOPPED due to quota even though profiling was simply disabled/sampled out; only check quota for an active launch profile or a session selected for continuous profiling.

Useful? React with 👍 / 👎.

@codecov-commenter

codecov-commenter commented Jul 9, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 83.11367% with 205 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.17%. Comparing base (d77dff5) to head (545541d).

Files with missing lines Patch % Lines
...dog/android/profiling/internal/ProfilingFeature.kt 82.89% 7 Missing and 19 partials ⚠️
...datadog/android/profiling/internal/NoOpProfiler.kt 13.79% 25 Missing ⚠️
...atadog/android/internal/profiling/ProfilerEvent.kt 0.00% 22 Missing ⚠️
.../android/profiling/internal/ProfilingDataWriter.kt 86.88% 16 Missing and 5 partials ⚠️
...iling/internal/anr/AnrProfilingTriggerRegistrar.kt 77.78% 9 Missing and 7 partials ⚠️
...profiling/internal/ContinuousProfilingScheduler.kt 92.25% 8 Missing and 3 partials ⚠️
...roid/insights/internal/DefaultInsightsCollector.kt 0.00% 8 Missing ⚠️
...nsights/internal/overlay/DefaultInsightsOverlay.kt 0.00% 8 Missing ⚠️
...roid/profiling/internal/utils/PackageManagerExt.kt 22.22% 6 Missing and 1 partial ⚠️
...id/internal/profiling/ProfilingAnrDetectedEvent.kt 0.00% 6 Missing ⚠️
... and 16 more
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #3622      +/-   ##
===========================================
+ Coverage    72.88%   73.17%   +0.29%     
===========================================
  Files          975      996      +21     
  Lines        35299    36325    +1026     
  Branches      5973     6129     +156     
===========================================
+ Hits         25726    26580     +854     
- Misses        7907     8032     +125     
- Partials      1666     1713      +47     
Files with missing lines Coverage Δ
...kotlin/com/datadog/android/core/InternalSdkCore.kt 100.00% <ø> (ø)
...in/java/com/datadog/android/profiling/Profiling.kt 81.82% <100.00%> (ø)
...atadog/android/profiling/ProfilingConfiguration.kt 100.00% <100.00%> (ø)
...com/datadog/android/profiling/internal/Profiler.kt 100.00% <100.00%> (ø)
...d/profiling/internal/ProfilingLifecycleCallback.kt 100.00% <100.00%> (ø)
...roid/profiling/internal/ProfilingRequestFactory.kt 100.00% <100.00%> (ø)
...roid/profiling/internal/perfetto/PerfettoResult.kt 100.00% <100.00%> (ø)
...og/android/profiling/internal/quota/QuotaReason.kt 100.00% <100.00%> (ø)
...og/android/profiling/internal/quota/QuotaResult.kt 100.00% <100.00%> (ø)
...ling/internal/telemetry/ProfilingTelemetryEvent.kt 100.00% <100.00%> (ø)
... and 32 more

... and 25 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ambushwork
ambushwork requested a review from kikoveiga July 9, 2026 10:02
@ambushwork
ambushwork merged commit 4f906ed into develop Jul 9, 2026
27 checks passed
@ambushwork
ambushwork deleted the yl/merge-develop-into-feature-profiling branch July 9, 2026 10:09
@ambushwork ambushwork changed the title Merge develop into feature/continuous-profiling Merge feature/continuous-profiling into develop Jul 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants