Skip to content

fix(stats): correct CI formula, zAxis drop, and edge guards#160

Merged
fahimfaisaal merged 1 commit into
mainfrom
fix/stats-math-review
Jun 26, 2026
Merged

fix(stats): correct CI formula, zAxis drop, and edge guards#160
fahimfaisaal merged 1 commit into
mainfrom
fix/stats-math-review

Conversation

@fahimfaisaal

Copy link
Copy Markdown
Member

Summary

Math review of ui/src/lib/stats.ts found several issues. This PR fixes the real bugs and tightens edge guards, keeping the stats worker dependency-free and lean.

Fixes

Bugs

  • computeProfiles dropped zAxis for descriptive stats — 3D charts silently collapsed across the z dimension while the correlation in the same bundle used z. Now passes zAxis through to computeDescriptive. (Regression test added.)
  • 95% CI was a z-interval (z=1.96) on the population SD — systematically too narrow for small n. Replaced with a Student-t interval: SEM uses the sample SD (n-1) and the critical value is t*_{n-1} from a compact lookup table with 1/df linear interpolation (<0.15% max error across df=1…1000, folding to z=1.95996 in the limit). CI is now NaN for n<2.

Edge guards

  • correlationMatrix diagonal is now NaN for constant (zero-spread) columns instead of a misleading 1, consistent with the NaN those columns produce off-diagonal.
  • cqv guard changed from p25+p75 !== 0 to > 0 so negative-spanning distributions return NaN instead of an unstable/negative ratio.

Docs

  • selectCorrelationAxis comment corrected (picks the smallest usable axis, not "prefer x").
  • mad doc notes it is unscaled (no 1.4826 factor) — not a σ-equivalent.

Approach note on the t-quantile

An initial implementation used Lanczos log-gamma + Lentz continued-fraction incomplete beta + bisection inverse (~95 lines, exact to 15 digits). That was over-engineered for a UI stats panel — replaced with a 12-row lookup table + 1/df linear interpolation (~40 lines, <0.15% max error). The error is invisible at UI render precision.

Infra fix

  • .lefthook.yml ui-gen-check hook used set -euo pipefail (bash-only); failed under sh (dash). Changed to set -eu.

Test plan

  • vitest run — 354/354 pass (78 in stats file, 4 new)
  • vue-tsc -b --noEmit — clean
  • prettier --check — clean
  • UI embed rebuilt; verify-gen-sync.ts confirms vizb-ui.gen.go in sync
  • All pre-commit hooks pass (format, go-test, ui-gen-check)

Math review of ui/src/lib/stats.ts found several issues:

- computeProfiles dropped zAxis for the descriptive piece, so 3D charts
  silently collapsed across the z dimension while correlation used z.
  Now passes zAxis through to computeDescriptive (regression test added).
- 95% CI was a z-interval (z=1.96) on the population SD — systematically
  too narrow for small n. Replaced with a Student-t interval: SEM uses
  the sample SD (n-1) and the critical value is t*_{n-1} from a compact
  lookup table with 1/df linear interpolation (<0.15% max error across
  df=1..1000, folding to z=1.95996 in the limit). CI is NaN for n<2.
- correlationMatrix diagonal is now NaN for constant (zero-spread)
  columns instead of a misleading 1, consistent with the NaN those
  columns produce off-diagonal.
- cqv guard changed from p25+p75 !== 0 to > 0 so negative-spanning
  distributions return NaN instead of an unstable/negative ratio.
- selectCorrelationAxis doc comment corrected (picks smallest usable
  axis, not 'prefer x'); mad doc notes it is unscaled (not sigma-equiv).

The over-engineered Lanczos/incomplete-beta t-quantile was replaced by
the compact table+interpolation to keep the stats worker lean.

Also fix .lefthook.yml ui-gen-check hook: `set -euo pipefail` is
bash-only and fails under sh (dash); use `set -eu` instead.
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@fahimfaisaal
fahimfaisaal merged commit d3a7b77 into main Jun 26, 2026
13 checks passed
@fahimfaisaal
fahimfaisaal deleted the fix/stats-math-review branch June 26, 2026 05:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants