Skip to content

physverify

Confirm physics claims by running the code: a suspected bug becomes a run against a reference from the physics, and a verdict with the numbers.

A simulation can run, draw nice plots and still be wrong: a minus sign in the wrong place, a factor of 2 missing, a time step that is off by one. Ordinary tests often miss these, because nobody knows the exact answer for a real simulation. physverify uses standard ideas from numerical analysis to check such code anyway, and explains each result in plain words.

pip install physverify
physverify demo

A claim and a reference from the physics go into the smallest run that tells them apart; the run gives a verdict

Run it on every push: add the GitHub Action to your workflow; a failed check fails the job.

      - uses: rcgiri-physics/[email protected]
        with:
          specs: tests/verify_*.py

Before trusting a PASS, read What a PASS means: the checks are only as independent as the reference equations you write.

What it checks

Check The question it answers Example
Manufactured solutions Does the code solve these equations? A midpoint method with a half-step typo
Order of convergence Does the error shrink as fast as the method promises? Hand in your own step sizes and errors
Reference solutions Does it agree with an independent, more accurate solution? A pendulum
Conserved quantities and budgets Is energy kept? Does heat leak through an insulated wall? Velocity Verlet, a 1D heat solver
Exact patch tests Does it reproduce a solution it should get exactly? A stage evaluated at the wrong time
Runge-Kutta tables Do the coefficients have the order and stability their name says? A table labelled order 3 that is order 2
Exact references Does it match a known closed-form result? Morse oscillator energy levels
Statistics Do the error bars mean what they say? Is a Monte Carlo sampler correct? 68% intervals that cover 30%
A suspected bug Is this claim from a review (yours or an AI's) true? A power spectrum deconvolved by W instead of W²

Every example on this site is a small script in docs/examples/, and the output shown is what it printed.

What a result looks like

mms        PASS   midpoint damped oscillator: observed order 2.04 (expected 2)
reference  PASS   midpoint damped oscillator: observed order 2.01 (expected 2)
mms        PASS   midpoint damped oscillator (whole trajectory): observed order 2.02 (expected 2)
patch      EXACT  midpoint damped oscillator: reproduced to round-off (largest error 2.7e-15), as designed
4 of 4 checks passed

Each check reports PASS, FAIL, EXACT (reproduced to round-off), INCONCLUSIVE (the errors could not be measured; the message says why), ERROR (the code crashed) or SKIP (an optional package is missing).

Does it work?

  • Confirming review claims (100 repositories, 2026-11): five AI reviewers reviewed 100 public physics repositories. On the 50 that earlier testing had found clean, 119 review claims were each checked by a run: 40 changed a result (33 distinct defects in 19 repositories), 51 were real but minor, 10 were wrong and 18 could not be settled. So review finds real defects that testing missed, and a run is what tells them from the wrong ones. Counts within this sample only; see Checking an AI code review.

Of 119 claims checked on repositories tested clean: 40 changed a result, 51 were real but minor, 10 were wrong, 18 stayed open

  • Seeded bugs: of 37 bugs added one at a time to five small solvers, it caught 35, with no false alarms on the bug-free versions. The bugs and the checks were written by the same person, so this shows the tool works as designed; it is not a promise about other code.
  • Public repositories: it was tried on more than 500 public physics repositories in a locked-down container and found confirmed bugs in some of them; see Results on real code.

Where to go next

Citing

If physverify helps your work, please cite it: doi:10.5281/zenodo.23534512 (all versions; each release also has its own DOI on Zenodo).

physverify is actively evolving (version 0.8.0), welcoming contributions and test cases from the computational physics community. Status: under active development and maintenance, tested on a growing corpus of public physics codes; releases come at milestones. See how to help and GitHub issues.