Tracking issue: short-term performance targets #4423
Description
The purpose of this issue is to list some performance targets and how close we are to reaching them.
This is the target we strive to reach on a modern laptop, e.g. a M1 MacBook Pro.
We currently lack benchmarks for most of these targets.
We know we are wildly off the mark on many of them.
Per-thread logging frequency
f32 scalar with one custom timeline.
Rust and C++: 1us per scalar = 1Mhz
Python: 100us per scalar = 10kHz
Status: We are off by ~3x at the moment.
Text logging should be similar.
Multi-threaded logging
Switching from one logging thread to N logging threads (N≤cores) should have at most 20% overhead, on any SDK.
Status: We know that we have problem with lock contention of Python
Video
- 4k video at 60 Hz RAW
- 4k video at 60 Hz with per-frame compression
The per-frame compression could e.g. be JPEG, JPEG2000, …
Status: Our SDK-side compressor is too slow, and it is also not multi-threaded
Latency
- No noticeable latency when streaming a web camera (<100ms)
Status: we're good
Data overhead
On-disk overhead and in-RAM overhead:
- 2x overhead: pain-limit
- 20% overhead: target
Overhead is measures as memory use divided by size of raw data (log data logged data + size of timeline values), ignoring caches.
Status: Right now our memory overhead for small datatypes (e.g. scalars) is enormous, because of arrow2 schema overhead.
Viewer performance
60 Hz when viewing:
- A static point cloud with custom colors, 1 million points large
- 60Hz 4k uncompressed video
- 60Hz 4k compressed video
- 10 time-scalar plots, with 10 lines per plot, each at 1kHz, viewing 1000 seconds of history