Skip to content

RUM-1662: Integration of SessionEndedMetric into sdk core#2090

Merged
ambushwork merged 1 commit into
feature/session-ended-metricfrom
yl/inject-session-ended-telemetry
Jun 20, 2024
Merged

RUM-1662: Integration of SessionEndedMetric into sdk core#2090
ambushwork merged 1 commit into
feature/session-ended-metricfrom
yl/inject-session-ended-telemetry

Conversation

@ambushwork

Copy link
Copy Markdown
Member

What does this PR do?

  • Integration of SessionEndedMetricDispatcher into core of SDK
  • Add integration tests to test "was_stopped" and view counts can be calculate correctly

Motivation

https://datadoghq.atlassian.net/browse/RUM-1662

Additional Notes

  • the way of tracking sdk error is modified, the session Id retrieved inside TelmetryEventHandler is always NULL_UUID, so now it relies on the last session id in SessionEndedMetricDispatcher
  • StubInternalLogger is updated to send the event when logMetric is called, so that in integration test the event can be intercepted.

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Make sure you discussed the feature or bugfix with the maintaining team in an Issue
  • Make sure each commit and the PR mention the Issue number (cf the CONTRIBUTING doc)

@ambushwork
ambushwork requested review from a team as code owners June 19, 2024 10:56

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change in this file needs to be reverted, OpenTelemetry is presented in the runtime configuration

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reverted

errorKindFrequencies[sdkErrorKind] = (errorKindFrequencies[sdkErrorKind] ?: 0) + 1
fun onErrorTracked(sdkErrorKind: String?) {
val errorKindKey = sdkErrorKind ?: SDK_ERROR_DEFAULT_KIND
errorKindFrequencies[errorKindKey] = (errorKindFrequencies[sdkErrorKind] ?: 0) + 1

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tiny mistake here, I believe

Suggested change
errorKindFrequencies[errorKindKey] = (errorKindFrequencies[sdkErrorKind] ?: 0) + 1
errorKindFrequencies[errorKindKey] = (errorKindFrequencies[errorKindKey] ?: 0) + 1

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

internal const val SDK_ERRORS_COUNT_BY_KIND_KEY = "by_kind"

/**
* Place holder of error kind if the attribute is absent.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Place holder of error kind if the attribute is absent.
* Placeholder of error kind if the attribute is absent.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we have some verification added here to check that onViewTracked is called (same for RumSessionScopeTest)? Or you want to rely solely on the integration tests?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

imo, M receive an event with correct view counts W track multiple views in the integration test can make sure that onViewTracked is called, otherwise it won't have the correct view count

}
}

override fun onSdkErrorTracked(sessionId: String, errorKind: String?) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cannot we have a situation when we stopped session A and started session B, but due to async nature of different pipelines in the SDK, we have this method called with error coming from session A by the time startMetric was called with Session B? not sure what is the probability of that though.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure the probability of this neither, but with this pull request, the error coming from Session A will belongs to Session B since startMetric has called. I am not sure if we have anyway to improve this, since endMetric of Session A has been called, the telemetry is sent out, the error will not be able to catch the train which has left.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory we can have a skew in the metric data we receive in such case, so maybe it is better to rollback this change and still pass sessionId to this method from the DatadogContext we have in the TelemetryEventHandler?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have two questions over this:

  1. even if we use session Id to track the error, the error count in session B can be correct, but in the session A will always be less than the correct one since the metric can be sent out, is that something we can tolerate?
  2. I tried to use sdkCore.getFeature(Feature.RUM_FEATURE_NAME)?.withWriteContext that you suggested to retrieve the session id for the error, I got always NULL_UUID, is that normal?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Don't know honestly, depends on the further metric usage and if we can have falsy data because of that.
  2. You don't need to use it directly, you can hook inside the lambda in the TelemetryEventHandler or you can use something like RumContext.fromFeatureContext(sdkCore.getDatadogContext().featuresContext[Feature.RUM_FEATURE_NAME]). NULL_UUID means that session is not yet created.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change is reverted, now we rely on the session id for the error tracking again

}

@Test
fun `M receive an event with "was_stopped" is true W stopSession() is called`(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

justification: W methodCall() already implies we are doing method call

Suggested change
fun `M receive an event with "was_stopped" is true W stopSession() is called`(
fun `M receive an event with "was_stopped" is true W stopSession()`(

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

}

@Test
fun `M receive an event with correct view counts W tracks multiple views`(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
fun `M receive an event with correct view counts W tracks multiple views`(
fun `M receive an event with correct view counts W track multiple views`(

Comment on lines +87 to +88
rumMonitor.startView(key = "key1", name = "View1")
rumMonitor.stopView(key = "key1")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

your test is saying that you are tracking multiple views, but here it is the same view: same key and same name. Yes, under the hood it will be indeed multiple views, but to make test clear it is better to use different keys and names:

Suggested change
rumMonitor.startView(key = "key1", name = "View1")
rumMonitor.stopView(key = "key1")
val viewKey = forge.anAlphaNumericalString()
rumMonitor.startView(key = viewKey, name = forge.anAlphaNumericalString())
rumMonitor.stopView(key = viewKey)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point, fixed

Comment on lines +104 to +97
private fun checkAttributeWithValue(map: Map<*, *>, actualKey: String, actualValue: Any): Boolean {
map.entries.forEach {
val entryValue = it.value
if (entryValue is Map<*, *>) {
return checkAttributeWithValue(entryValue, actualKey, actualValue)
} else if (it.key == actualKey) {
return it.value == actualValue
}
}
return false
}

@0xnm 0xnm Jun 19, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this may make your test green in case of the bug. Imagine that you expect attribute buzz to be under full path foo.bar.buzz, but there is something in the model that has also path foo.buzz. Then your test will be green even if foo.bar.buzz is missing.

Check internals of RumEventAssert as an example, you can add the necessary method there and rely on the full path validation.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you are right, it's fixed

Comment on lines +50 to +58
val rumFeature = sdkCore.getFeature(RUM_FEATURE_NAME) ?: return
val message = messageBuilder()
val telemetryEvent =
mapOf(
"type" to "mobile_metric",
"message" to message,
"additionalProperties" to additionalProperties
)
rumFeature.sendEvent(telemetryEvent)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is quite dangerous, you can cause infinite recursion here if somewhere in the RumFeature.sendEvent call stack there is call for the logger metric as well.

I think it will be simpler here just to record the data received in the property of StubInternalLogger and then query it.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now I keep a lastMetric inside StubInternalLogger to store the telemetry event

@ambushwork
ambushwork force-pushed the yl/inject-session-ended-telemetry branch 5 times, most recently from 3726630 to 5d0ed75 Compare June 20, 2024 08:16
@ambushwork
ambushwork requested a review from 0xnm June 20, 2024 12:47
rumMonitor.stopSession()

// Then
RumEventAssert.assertThat(stubSdkCore.lastMetric())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: you can also use static import for RumEventAssert.assertThat here.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Comment on lines +142 to +144
fun assertThat(actual: Any): RumEventAssert {
return assertThat(Gson().toJsonTree(actual).asJsonObject)
}

@0xnm 0xnm Jun 20, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

blocking: IMO this looks like a hack.

Ideally, I don't think it should be Any argument type here, and probably this shouldn't be inside RumEventAssert. RumEventAssert is used for the RUM events only (the ones which are going to be sent to the intake), which are RUM View, RUM Error, etc. events.

In your case you have argument which is not a RUM event (and obviously none of the RUM events can have "wasStopped" or "hasViewCount" attribute). This approach works, but it is mixing the things. Maybe it is better rather to create another assertion class (say for Map<String, Any> or even Map<*,*>) and since you are not operating with JSON you can write something like that there (let's assume actual is of type Map<String, Any>):

fun hasWasStopped(wasStopped: Boolean): TelemetryMetricAssert {
        assertThat(actual).containsPathEntry(path, value)
        return this
    }

and then you just need to implement containsPathEntry which will take the path like additionalProperties.rse.views_count.total in a similar manner how it was done for json (just split in the fragments and iterate over).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TelemetryMetricAssert class is now created for this, please check it

Comment on lines 16 to 17
var lastMetric: Map<String, Any> = hashMapOf()
override fun log(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

following approach of capturing all the data in the test, it is probably more useful to record all the events (and then access the necessary using last() or findLast, etc.), also making this property immutable

Suggested change
var lastMetric: Map<String, Any> = hashMapOf()
override fun log(
val telemetryEventsWritten = mutableListOf<Map<String, Any>>()
override fun log(

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

@ambushwork
ambushwork force-pushed the yl/inject-session-ended-telemetry branch from 5d0ed75 to 88200c2 Compare June 20, 2024 15:11
@ambushwork
ambushwork requested a review from 0xnm June 20, 2024 15:12
0xnm
0xnm previously approved these changes Jun 20, 2024
* Returns the last metric is sent by [StubInternalLogger].
*/
fun lastMetric(): Map<String, Any> {
return (internalLogger as StubInternalLogger).telemetryEventsWritten.last()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to be precise, you need to use filter { it["type"] == "mobile_metric" } here, because property itself is called telemetryEventsWritten, so if someone else adds there non-metric telemetry, telemetryEventsWritten.last() will give a wrong result and things won't be correct.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sense, I use lastOrNull{ it["type"] == "mobile_metric"} to return the last telemetry metric recorded.

@Suppress("UnsafeThirdPartyFunctionCall")
internal class StubInternalLogger : InternalLogger {

var telemetryEventsWritten = mutableListOf<Map<String, Any>>()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need to be mutable reference.

Suggested change
var telemetryEventsWritten = mutableListOf<Map<String, Any>>()
val telemetryEventsWritten = mutableListOf<Map<String, Any>>()

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@ambushwork
ambushwork force-pushed the yl/inject-session-ended-telemetry branch from 88200c2 to 89b0892 Compare June 20, 2024 15:34
@ambushwork
ambushwork requested a review from 0xnm June 20, 2024 15:35
@ambushwork
ambushwork merged commit f2aa3d8 into feature/session-ended-metric Jun 20, 2024
@ambushwork
ambushwork deleted the yl/inject-session-ended-telemetry branch June 20, 2024 19:11
@xgouchet xgouchet added this to the 2.12.x milestone Jul 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants