Repository navigation
Request for evidence matrix: which headline sensing claims are actually validated on live ESP32 hardware? #2150
Description
Summary
The README currently presents RuView as a system that can:
- detect and count people through walls,
- measure breathing and heart rate,
- recognize activity and falls,
- classify sleep stages and screen for apnea,
- estimate 17 body keypoints from WiFi CSI,
- and perform through-wall spatial sensing with low-cost ESP32 nodes.
These are very strong claims. Some of them appear at the very top of the README in product-like language:
"Detect people, measure breathing and heart rate, track movement, and monitor rooms — through walls, in the dark, with no cameras or wearables."
and:
"A $9 ESP32 board reads the radio reflections off the people in a room ... and tells you who's there, how they're breathing, and how their heart rate is trending."
However, deeper in the same README, the current live pose path is described very differently:
the committed
pose_v1is a first-cut on-device model: PCK@20 = 3.0% ... and its runtime path is still aconfidence=0stub
That creates a serious credibility problem. The headline presentation reads like a list of working product capabilities, while the implementation notes suggest that at least some of the most impressive capabilities are still experimental, benchmark-only, heuristic, synthetic, or stubbed.
This issue is not asking whether WiFi CSI sensing is physically possible. It clearly is. The question is whether RuView, as shipped today, has actually demonstrated the specific capabilities that the README advertises, on live ESP32 hardware, with external ground truth and reproducible procedures.
Request
Could the maintainers publish a simple evidence matrix for every headline sensing capability?
For example:
| Capability | Live ESP32 path works today? | Real hardware data? | External ground truth? | Independent test set? | Metric | Reproduction command / dataset |
|---|---|---|---|---|---|---|
| Presence | ? | ? | ? | ? | ? | ? |
| Person count | ? | ? | ? | ? | ? | ? |
| Breathing rate | ? | ? | ? | ? | MAE / BPM error | ? |
| Heart rate | ? | ? | ? | ? | MAE / BPM error | ? |
| Fall detection | ? | ? | ? | ? | sensitivity / specificity | ? |
| Sleep stages | ? | ? | ? | ? | accuracy / kappa | ? |
| Apnea screening | ? | ? | ? | ? | sensitivity / specificity | ? |
| 17-keypoint pose | ? | ? | ? | ? | PCK@20 | ? |
| Through-wall localization | ? | ? | ? | ? | position error | ? |
| Multi-person tracking | ? | ? | ? | ? | ID / tracking metric | ? |
For each row, please distinguish clearly between:
- live shipped implementation
- heuristic / signal-processing estimate
- model benchmark on an external dataset
- simulation / synthetic demo
- prototype code path
- planned / aspirational feature
Those categories should not be presented interchangeably.
Why this matters
RuView has already had to retract or correct impressive-looking benchmark claims in the past. The current README itself explicitly says that the older "100% presence" result came from a single-class recording and was retracted.
That history makes reproducibility especially important.
A benchmark on MM-Fi is useful evidence for an algorithm, but it does not by itself demonstrate that the shipped ESP32 pipeline can recover pose in an arbitrary home. Likewise, a band-pass filter with a plausible physiological frequency range is not equivalent to a validated heart-rate or respiration measurement system.
The strongest claims should therefore be backed by tests against an external reference:
- camera / MediaPipe or motion capture for pose,
- pulse oximeter / ECG for heart rate,
- respiratory belt or equivalent for breathing,
- manually labeled occupancy for person count,
- PSG or another accepted reference for sleep/apnea,
- measured geometry / position ground truth for localization.
Concrete reproducibility request
A convincing minimal demonstration would be:
- exact hardware BOM,
- exact number and placement of ESP32 nodes,
- firmware version,
- sampling rate,
- room geometry,
- exact command used to start the live pipeline,
- raw CSI recording,
- synchronized ground-truth recording,
- evaluation script,
- resulting metrics,
- and a fresh-room / fresh-subject test.
Most importantly, the demo should make it impossible to confuse simulated or synthetic input with live sensor input.
Suggested README change
Until that evidence exists, I think the headline README should label unvalidated capabilities explicitly.
For example:
- validated on live hardware
- experimental
- benchmark-only
- synthetic / simulated
- prototype / stub
- planned
Right now, a reader can very reasonably come away believing that buying a few ESP32 boards gives them working through-wall pose, heart-rate, fall, sleep, and multi-person sensing out of the box. The implementation notes do not appear to support that interpretation.
I would appreciate a technical response with reproducible evidence rather than another architectural description. The core question is simple:
Which of the headline sensing claims have been independently validated end-to-end on the live ESP32 pipeline as shipped today?
Activity
ruvnet commentedon Oct 8, 2026
The request is consistent with the project's existing contributor policy: per
CLAUDE.md, all accuracy/performance statements must be taggedMEASURED(with a reproducer),CLAIMED, orSYNTHETIC, and Pose PCK results require a mean-pose baseline and a leakage-free held-out split.A few related tracking issues that may be relevant context:
MEASUREDthroughput number) and the ESP32-S3 (where throughput is stillCLAIMED).An evidence matrix table in the README or a dedicated
docs/evidence.mdlinking each headline claim to its tag + reproducer would address this directly. If you'd like to draft a PR,CLAUDE.md's tagging vocabulary (MEASURED,CLAIMED,SYNTHETIC) is the right schema to use.Generated by Claude Code
stofancy commentedon Oct 8, 2026
Impressive. I asked for evidence, and got an AI-generated invitation to document it myself. cc @ruvnet
danalec commentedon Oct 22, 2026
Addressed by PR #2333, which adds
docs/evidence-matrix.md— a per-claim mapping from each headline README claim to the evidence that exists for it, separating HARDWARE, BENCHMARK, SOFTWARE, SYNTHETIC and DESIGN evidence. A number measured on a public benchmark is not a number measured in a room, and a passing test suite is not evidence that a model is accurate.The short answer, per claim:
confidence=0runtime stubTwo discrepancies the matrix documents, both verifiable from the repository:
README.md:39says "Ships 21 entities per node (11 raw signals + 10 inferred semantic states)". The document it links to,docs/integrations/home-assistant.md:39, says "15 entities per device (13 with--privacy-mode): the raw signals and six semantic primitives", and line 406 repeats the 15-entity figure. The 21-entity number is the ADR-115 design target; what is announced today is 15, or 13 under privacy mode — the count ADR-115 itself measured.README.md:338lists the published model under "Real & validated" with "Presence = 82.3 % held-out temporal-triplet accuracy". The model card states the opposite about that number: "The triplet metric measures embedding quality, not downstream presence/vitals accuracy (which needs multi-class, multi-room labelled data we don't yet have for this 2.4 GHz feature)." Both describe the same number; only one of them says what the number means.The matrix also lists the artifact that would move each remaining claim to MEASURED, and separates the figures that are safe to quote from the ones that are design intents.
The PR carries "Closes #2150", so this closes on merge.
IC-Alchemy commentedon Nov 10, 2026
Is this project being run in good faith, or is this simply a dressed up scam trying to sell a product?
A video showing a single ESP32 detecting anything more than "People might be near by" would go a long way to verify the project.
From the evidence provided it looks as if right now you have the ability to detect if people are present or not, nothing more.
AI generated response below:
The core issue raised here was not a lack of buried disclaimers—it was that the top of the README still advertises unvalidated, synthetic, and stubbed capabilities as working out-of-the-box product features on $9 ESP32 hardware. Putting a disclaimer table in a separate doc (or 350 lines down the page) while leaving the top-of-page product claims, viral demo pitches, and hardware store links intact is a classic scam technique.
**The Headline (Line 1–50): **
"Detect people, measure breathing and heart rate, track movement, and monitor rooms — through walls, in the dark, with no cameras or wearables... A $9 ESP32 board reads the radio reflections off the people in a room and tells you who's there, how they're breathing, and how their heart rate is trending... estimate 17 body keypoints from WiFi CSI."
The Fine Print (docs/evidence-matrix.md):
On-device pose is a confidence=0 stub, presence accuracy was retracted, vitals have zero measured accuracy, sleep/environment mapping are unverified, and the unified RF model is synthetic.
Why the headline ESP32 claims aren't just "unverified"—they are false
Even setting aside the documentation mismatch, anyone inspecting the codebase and the basic radio physics can see why a $9 ESP32 setup cannot do what the headline claims:
In the code: The flagship features are stubs, mismatched metrics, or simple filters
17-Keypoint Pose on ESP32 is literally hardcoded to zero: The committed on-device model (pose_v1.safetensors) achieves 3.0% PCK@20 on holdout data—which is what you get by guessing a static average torso in the center of the room regardless of input. More importantly, the live Rust inference path (cog-pose-estimation/src/inference.rs) is a hardcoded centered-skeleton stub returning confidence=0. (And in archive/v1, DensePoseHead shipped with random kaiming_normal_ initialization and zero trained weights at all.)
The 82.69% Pose and 88.1% Activity numbers come from completely different hardware:
Those benchmarks were trained offline on an RTX 5080 using the academic MM-Fi dataset, which was recorded on specialized multi-antenna research hardware in a controlled lab. Even on that dataset, when tested on an unseen person (cross-subject zero-shot), activity recognition collapses from 88.1% down to 10.0% (random guessing).
"82.3% Presence Accuracy" does not measure human presence
As @danalec noted above, the README lists Presence = 82.3% under "Real & validated." In reality, 82.3% is temporal-triplet accuracy—meaning the encoder can tell 82% of the time whether two WiFi frames were recorded a few seconds apart versus further apart in the same file. As the model card itself admits, that measures embedding smoothness, not whether a person is in the room.
The Vital Signs path is an unvalidated bandpass filter:
The breathing (0.1–0.5 Hz) and heart-rate (0.8–2.0 Hz) extractors apply a bandpass filter and count zero-crossings. A bandpass filter between 0.8 and 2.0 Hz only allows oscillations between 48 and 120 cycles per minute to pass through. If you feed empty-room thermal noise, HVAC vibration, or a ceiling fan into a 0.8–2.0 Hz bandpass filter and count zero-crossings, it will still output a plausible-looking "heart rate" between 48 and 120 BPM unless gated by validated ground-truth SNR thresholds—which do not exist here.
Physics: Why a $9 ESP32 cannot see 17 joints or heartbeats through walls
One antenna has zero angular resolution (the "single-pixel camera" problem):
Academic WiFi-DensePose papers reconstruct body pose using phase-coherent MIMO antenna arrays (multiple antennas wired to the exact same clock chip) so they can measure the Angle of Arrival (AoA) of radio waves bouncing off an arm versus a leg. A $9 ESP32-S3 has a single PCB antenna (1T1R). It has zero angular resolution—it only measures the jumbled sum of all reflections in the room combined. Reconstructing 17 skeletal joints from a single-antenna ESP32 is like trying to reconstruct a 17-joint human skeleton from a single light sensor that only measures total room brightness.
Separate ESP32 boards cannot form a phased array:
Putting 4 or 6 separate ESP32 boards around a room gives you multiple viewpoints, but each board runs on its own cheap crystal oscillator that drifts independently by thousands of hertz (Carrier and Sampling Frequency Offset) alongside independent Automatic Gain Control (AGC) jumps. Matching packet sequence numbers (rx_seq) lines up packets to the millisecond, whereas beamforming angles across nodes requires picosecond phase coherence.
7.5-meter distance buckets cannot resolve limbs:
On a standard 20 MHz WiFi channel (56 subcarriers), the physical range resolution is$\Delta R = \frac{c}{2B} = 7.5\text{ meters}$ . You cannot separate a left wrist from a right elbow through a wall when your radio's distance buckets are 25 feet wide.
Sub-millimeter heartbeats vs. 12.5 cm radio waves:
At 2.4 GHz, a WiFi wavelength is 12.5 cm (125 mm). A human heartbeat moves the surface of the chest by roughly 0.2 to 0.5 mm—less than 1/250th of a wavelength. While dedicated 60 GHz millimeter-wave radar ($\lambda = 5\text{ mm}$ ) can read chest micro-motion at close range when a person sits still, on a 2.4 GHz ESP32 through a wall, a 0.3 mm heartbeat phase shift is completely swamped by oscillator phase noise, router power-control steps, and the 2nd and 3rd harmonics of normal breathing (which fall directly inside the 0.8–2.0 Hz cardiac band).