Skip to content

One CPU fully occupied by Datadog when device offline #1469

Description

@torbjorn-humane

We're using the Datadog android java SDK to collect both traces and logs on an android device. The device may be offline for an extended period, which means logs/traces cannot be uploaded but are stored on device. After a while in this state, the datadog agent is fully occupying 1 CPU. We have reproduced the issue both on device and using the android emulator. We have tested both version 1.9.0 and 1.19.2 and the issue persists. We're using the Datadog-Timber integration for log/trace capturing.

Steps to reproduce the issue:

  1. Create an android instrumented test that loops over a log statement say 100k times.
  2. Start the android emulator and enable airplane mode on the emulated device.
  3. Run the test and observe the folder on device (using e.g. adb shell) where the cached files are stored (e.g. /data/user/0/<myapplication>/cache/datadog-.../logs-v2
  4. When the number of cached files reaches ~200, stop the test.
  5. Create another instrumented test that logs in a loop. For each iteration of the loop, sleep 10ms (emulating 100 log entries/second).
  6. Run the second test and attach the profiler, e.g. android studio. One of the worker threads will occupy one CPU 100%. Breaking in the debugger indicates that the Datadog agent is using one of the worker threads to iterate over every stored file repeatedly, causing the high CPU usage.

Describe what you expected:
Running the same test with datadog enabled, emulating 100 log writes/second, occupies ~8% of one CPU when the device has network access. I'd expect no more CPU usage even if the device is offline.

Additional context

  • Android OS version: 31
  • Emulator: Pixel 6 Pro
  • Datadog SDK version: 1.9.0, 1.19.2
  • Versions of any other relevant dependencies (OkHttp, …): Timber 5.0.1, okhttp 4.9.3
  • Proguard configuration: N/A
  • Gradle Plugins: N/A

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions