Skip to content

Calibration qualification can accept an always-absent model without an occupied sensitivity control #1952

Open

Description

@proffesor-for-testing

A completed empty-room calibration can pass startup-baseline promotion while failing every known-occupied control. The promotion result therefore cannot establish occupancy qualification.

MEASURED on a Mac Catalyst app with its bundled arm64 sensing-server at dbf47886289b344fff8c399d5ff17c4cdf3b4da3 (PR #1936, based on #1903), four ESP32-C6 sources, 2026-09-16:

  • Empty capture finalized with 19 baseline eigenvalues, active 256-subcarrier grid, no missing contributors.
  • Learned residual threshold: 47.9984.
  • Bootstrap promotion: 12/12 samples empty; baseline stored on disk.
  • Subsequent operator-labelled phases: sitting 0/865 calibrated-present, walking 0/927, sitting 0/786. Raw heuristic reported presence; raw output is not ground truth.

The operator-labelled walking control falsifies sensitivity. A detector that always returns zero trivially satisfies an empty-only check. Disturbed-baseline threshold inflation is a plausible mechanism addressed separately by #1917; this issue concerns the missing qualification control regardless of the mechanism.

Important contract distinction: stored bootstrap models intentionally have negative-only authority. This is not a request to grant them positive presence or vitals authority. Keep bootstrap storage/background checks separate from a claim that the explicit field calibration is qualified for occupancy.

Suggested acceptance:

  1. An always-absent model fails occupancy qualification against a known-occupied, operator-labelled held-out phase.
  2. Pair sensitivity with a sufficiently long independent empty phase and report sample durations, false alarms and missed detections separately.
  3. Match the exact boot/session/model/binding identity and require fresh CSI from the radio actually scored; refuse cached/stale evidence.
  4. The Mac UI must distinguish stored bootstrap baseline from occupancy-qualified field calibration.
  5. Include real app/server contract fixtures; schema literals and mocked frames alone do not establish producer/consumer compatibility.

Related: #1917, #1936, #1940. No production/fleet accuracy claim is made from this installation.

Activity

  1. added
    securitySecurity-related issues and hardening
    and removed
    securitySecurity-related issues and hardening
    on Sep 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions