RUM-1662: Integration of SessionEndedMetric into sdk core#2090
Conversation
There was a problem hiding this comment.
change in this file needs to be reverted, OpenTelemetry is presented in the runtime configuration
| errorKindFrequencies[sdkErrorKind] = (errorKindFrequencies[sdkErrorKind] ?: 0) + 1 | ||
| fun onErrorTracked(sdkErrorKind: String?) { | ||
| val errorKindKey = sdkErrorKind ?: SDK_ERROR_DEFAULT_KIND | ||
| errorKindFrequencies[errorKindKey] = (errorKindFrequencies[sdkErrorKind] ?: 0) + 1 |
There was a problem hiding this comment.
tiny mistake here, I believe
| errorKindFrequencies[errorKindKey] = (errorKindFrequencies[sdkErrorKind] ?: 0) + 1 | |
| errorKindFrequencies[errorKindKey] = (errorKindFrequencies[errorKindKey] ?: 0) + 1 |
| internal const val SDK_ERRORS_COUNT_BY_KIND_KEY = "by_kind" | ||
|
|
||
| /** | ||
| * Place holder of error kind if the attribute is absent. |
There was a problem hiding this comment.
| * Place holder of error kind if the attribute is absent. | |
| * Placeholder of error kind if the attribute is absent. |
There was a problem hiding this comment.
should we have some verification added here to check that onViewTracked is called (same for RumSessionScopeTest)? Or you want to rely solely on the integration tests?
There was a problem hiding this comment.
imo, M receive an event with correct view counts W track multiple views in the integration test can make sure that onViewTracked is called, otherwise it won't have the correct view count
| } | ||
| } | ||
|
|
||
| override fun onSdkErrorTracked(sessionId: String, errorKind: String?) { |
There was a problem hiding this comment.
cannot we have a situation when we stopped session A and started session B, but due to async nature of different pipelines in the SDK, we have this method called with error coming from session A by the time startMetric was called with Session B? not sure what is the probability of that though.
There was a problem hiding this comment.
I am not sure the probability of this neither, but with this pull request, the error coming from Session A will belongs to Session B since startMetric has called. I am not sure if we have anyway to improve this, since endMetric of Session A has been called, the telemetry is sent out, the error will not be able to catch the train which has left.
There was a problem hiding this comment.
In theory we can have a skew in the metric data we receive in such case, so maybe it is better to rollback this change and still pass sessionId to this method from the DatadogContext we have in the TelemetryEventHandler?
There was a problem hiding this comment.
I have two questions over this:
- even if we use session Id to track the error, the error count in session B can be correct, but in the session A will always be less than the correct one since the metric can be sent out, is that something we can tolerate?
- I tried to use
sdkCore.getFeature(Feature.RUM_FEATURE_NAME)?.withWriteContextthat you suggested to retrieve the session id for the error, I got always NULL_UUID, is that normal?
There was a problem hiding this comment.
- Don't know honestly, depends on the further metric usage and if we can have falsy data because of that.
- You don't need to use it directly, you can hook inside the lambda in the
TelemetryEventHandleror you can use something likeRumContext.fromFeatureContext(sdkCore.getDatadogContext().featuresContext[Feature.RUM_FEATURE_NAME]).NULL_UUIDmeans that session is not yet created.
There was a problem hiding this comment.
The change is reverted, now we rely on the session id for the error tracking again
| } | ||
|
|
||
| @Test | ||
| fun `M receive an event with "was_stopped" is true W stopSession() is called`( |
There was a problem hiding this comment.
justification: W methodCall() already implies we are doing method call
| fun `M receive an event with "was_stopped" is true W stopSession() is called`( | |
| fun `M receive an event with "was_stopped" is true W stopSession()`( |
| } | ||
|
|
||
| @Test | ||
| fun `M receive an event with correct view counts W tracks multiple views`( |
There was a problem hiding this comment.
| fun `M receive an event with correct view counts W tracks multiple views`( | |
| fun `M receive an event with correct view counts W track multiple views`( |
| rumMonitor.startView(key = "key1", name = "View1") | ||
| rumMonitor.stopView(key = "key1") |
There was a problem hiding this comment.
your test is saying that you are tracking multiple views, but here it is the same view: same key and same name. Yes, under the hood it will be indeed multiple views, but to make test clear it is better to use different keys and names:
| rumMonitor.startView(key = "key1", name = "View1") | |
| rumMonitor.stopView(key = "key1") | |
| val viewKey = forge.anAlphaNumericalString() | |
| rumMonitor.startView(key = viewKey, name = forge.anAlphaNumericalString()) | |
| rumMonitor.stopView(key = viewKey) |
| private fun checkAttributeWithValue(map: Map<*, *>, actualKey: String, actualValue: Any): Boolean { | ||
| map.entries.forEach { | ||
| val entryValue = it.value | ||
| if (entryValue is Map<*, *>) { | ||
| return checkAttributeWithValue(entryValue, actualKey, actualValue) | ||
| } else if (it.key == actualKey) { | ||
| return it.value == actualValue | ||
| } | ||
| } | ||
| return false | ||
| } |
There was a problem hiding this comment.
this may make your test green in case of the bug. Imagine that you expect attribute buzz to be under full path foo.bar.buzz, but there is something in the model that has also path foo.buzz. Then your test will be green even if foo.bar.buzz is missing.
Check internals of RumEventAssert as an example, you can add the necessary method there and rely on the full path validation.
There was a problem hiding this comment.
you are right, it's fixed
| val rumFeature = sdkCore.getFeature(RUM_FEATURE_NAME) ?: return | ||
| val message = messageBuilder() | ||
| val telemetryEvent = | ||
| mapOf( | ||
| "type" to "mobile_metric", | ||
| "message" to message, | ||
| "additionalProperties" to additionalProperties | ||
| ) | ||
| rumFeature.sendEvent(telemetryEvent) |
There was a problem hiding this comment.
this is quite dangerous, you can cause infinite recursion here if somewhere in the RumFeature.sendEvent call stack there is call for the logger metric as well.
I think it will be simpler here just to record the data received in the property of StubInternalLogger and then query it.
There was a problem hiding this comment.
now I keep a lastMetric inside StubInternalLogger to store the telemetry event
3726630 to
5d0ed75
Compare
| rumMonitor.stopSession() | ||
|
|
||
| // Then | ||
| RumEventAssert.assertThat(stubSdkCore.lastMetric()) |
There was a problem hiding this comment.
suggestion: you can also use static import for RumEventAssert.assertThat here.
| fun assertThat(actual: Any): RumEventAssert { | ||
| return assertThat(Gson().toJsonTree(actual).asJsonObject) | ||
| } |
There was a problem hiding this comment.
blocking: IMO this looks like a hack.
Ideally, I don't think it should be Any argument type here, and probably this shouldn't be inside RumEventAssert. RumEventAssert is used for the RUM events only (the ones which are going to be sent to the intake), which are RUM View, RUM Error, etc. events.
In your case you have argument which is not a RUM event (and obviously none of the RUM events can have "wasStopped" or "hasViewCount" attribute). This approach works, but it is mixing the things. Maybe it is better rather to create another assertion class (say for Map<String, Any> or even Map<*,*>) and since you are not operating with JSON you can write something like that there (let's assume actual is of type Map<String, Any>):
fun hasWasStopped(wasStopped: Boolean): TelemetryMetricAssert {
assertThat(actual).containsPathEntry(path, value)
return this
}
and then you just need to implement containsPathEntry which will take the path like additionalProperties.rse.views_count.total in a similar manner how it was done for json (just split in the fragments and iterate over).
There was a problem hiding this comment.
TelemetryMetricAssert class is now created for this, please check it
| var lastMetric: Map<String, Any> = hashMapOf() | ||
| override fun log( |
There was a problem hiding this comment.
following approach of capturing all the data in the test, it is probably more useful to record all the events (and then access the necessary using last() or findLast, etc.), also making this property immutable
| var lastMetric: Map<String, Any> = hashMapOf() | |
| override fun log( | |
| val telemetryEventsWritten = mutableListOf<Map<String, Any>>() | |
| override fun log( |
5d0ed75 to
88200c2
Compare
| * Returns the last metric is sent by [StubInternalLogger]. | ||
| */ | ||
| fun lastMetric(): Map<String, Any> { | ||
| return (internalLogger as StubInternalLogger).telemetryEventsWritten.last() |
There was a problem hiding this comment.
to be precise, you need to use filter { it["type"] == "mobile_metric" } here, because property itself is called telemetryEventsWritten, so if someone else adds there non-metric telemetry, telemetryEventsWritten.last() will give a wrong result and things won't be correct.
There was a problem hiding this comment.
Make sense, I use lastOrNull{ it["type"] == "mobile_metric"} to return the last telemetry metric recorded.
| @Suppress("UnsafeThirdPartyFunctionCall") | ||
| internal class StubInternalLogger : InternalLogger { | ||
|
|
||
| var telemetryEventsWritten = mutableListOf<Map<String, Any>>() |
There was a problem hiding this comment.
no need to be mutable reference.
| var telemetryEventsWritten = mutableListOf<Map<String, Any>>() | |
| val telemetryEventsWritten = mutableListOf<Map<String, Any>>() |
88200c2 to
89b0892
Compare
What does this PR do?
SessionEndedMetricDispatcherinto core of SDKMotivation
https://datadoghq.atlassian.net/browse/RUM-1662
Additional Notes
NULL_UUID, so now it relies on the last session id inSessionEndedMetricDispatcherStubInternalLoggeris updated to send the event whenlogMetricis called, so that in integration test the event can be intercepted.Review checklist (to be filled by reviewers)