Reuse the pose graph of GlobalPipeline in the component mappers - #4912

## Summary

`CovarianceWeightedCostFunction` leaked the wrapped cost function when
computing the square root information threw, and silently produced
non-finite weights for covariances that are not positive definite.

- The owning `std::unique_ptr` member is now declared, and thus
initialized, before the square root information, so the wrapped cost
function is released if that throws.
- A null cost function is rejected with `THROW_CHECK_NOTNULL`.
- The covariance must be positive definite (finite inverse and
successful Cholesky factorization), otherwise the constructor throws
`std::invalid_argument` instead of producing NaN weights.
- The square root information is computed by a shared helper,
`internal::ComputeLeftSqrtInformation`, which #4823 reuses for a
dynamic-size counterpart.

## Tests

`CovarianceWeightedCostFunction.RejectsNonPositiveDefiniteCovariance`
checks that zero and indefinite covariances, and a null cost function,
throw. Under ASan, it also checks that the wrapped cost function does
not leak.

This is the first PR of a stack: #4823 builds on it, and #4912 builds on
#4823.

## Summary

Estimates the 3×3 marginal covariance of a two-view relative rotation
from its inlier correspondences:

- For a calibrated or uncalibrated relative pose $`(R, t)`$ with $`\|t\|
= 1`$, `EstimateTwoViewRotationCovariance` evaluates the Jacobian $`J`$
of the tangent Sampson error over `geometry.inlier_matches` w.r.t. the
right perturbation $`R\,\mathrm{Exp}(\delta\theta)`$ and the tangent
perturbation of $`t \in S^2`$. This yields the 5-DoF information matrix

$$
\Lambda = \frac{1}{\sigma^2} J^\top J = \begin{pmatrix} \Lambda_{RR} &
\Lambda_{Rt} \\ \Lambda_{tR} & \Lambda_{tt} \end{pmatrix},
\qquad
\Sigma_R = \left(\Lambda_{RR} - \Lambda_{Rt} \Lambda_{tt}^{-1}
\Lambda_{tR}\right)^{-1} ,
$$

where the translation direction is marginalized with the Schur
complement and $`\sigma`$ is the 2D observation noise
(`point2D_stddev_px`, 1 px by default).
- Pure-rotation (panoramic) pairs use a rotation-only model on the 2D
tangent planes orthogonal to the second camera's rays.
- The standard deviation along poorly or unconstrained axes (e.g.,
geometrically degenerate configurations) is capped to
`max_rotation_sigma_deg` (60°), with a `1e-5` relative variance floor so
single-precision storage preserves positive-definiteness. Returns
`std::nullopt` if the relative pose is missing, has too few valid
inliers, or if the eigendecomposition of the information fails.
- The covariance is expressed in camera 1's local tangent frame,
matching `RelativeRotationCostFunctor`.
- Exposes `ExtractMatchedCamRaysWithJac` in
`estimators/two_view_geometry.h`.
- Adds pycolmap bindings: `TwoViewRotationCovarianceOptions` and
`estimate_two_view_rotation_covariance`.

## Tests

- `TwoViewRotationCovarianceTests.MonteCarloCalibration`: 1000-trial
Monte Carlo NEES ($`\delta\theta^\top \Sigma_R^{-1} \delta\theta`$)
checks against the rotation MLE across wide-baseline, narrow-baseline,
forward-motion, planar, one-sided, and pure-rotation configurations.
- `TwoViewRotationCovarianceOptions.Check`,
`EstimateTwoViewRotationCovariance.Nominal`,
`UnconstrainedAxisIsCapped`, and `MissingPoseOrTooFewInliers`.
- `ExtractMatchedCamRaysWithJac.Nominal`.
- Python binding tests in
`src/pycolmap/estimators/two_view_rotation_covariance_test.py`.

PR stack (1/5 splitting #4823, plus #4912):
1. **#4950** (this PR)
2. #4951
3. #4952
4. #4953
5. #4823
6. #4912

## Summary

Stores optional relative-rotation covariances on `PoseGraph::Edge` and
estimates them in parallel from a `DatabaseCache`:

- Adds `std::optional<Eigen::Matrix3f> cam2_from_cam1_rotation_cov` to
`PoseGraph::Edge` (single precision to keep edges lightweight),
expressed in camera 1's local tangent frame. `PoseGraph::Edge::Invert()`
conjugates it accordingly.
- Adds `EstimatePoseGraphRotationCovariances`, which populates
`cam2_from_cam1_rotation_cov` in parallel over all valid pose graph
edges that do not already have one, evaluated at each edge's relative
pose from its correspondence-graph inliers. Edges whose covariance
cannot be estimated are left unset. The database schema does not change.
- Adds pycolmap bindings: `PoseGraphEdge.cam2_from_cam1_rotation_cov`
and `estimate_pose_graph_rotation_covariances`.

## Tests

- `PoseGraph.InvertEdgeCovariance`: checks covariance conjugation and
round-trip under `Edge::Invert()`.
- `EstimatePoseGraphRotationCovariances.Nominal`, plus single-precision
inversion positive-definiteness in
`EstimateTwoViewRotationCovariance.UnconstrainedAxisIsCapped` and option
validation in `TwoViewRotationCovarianceOptions.Check`.
- Python binding tests in `pose_graph_test.py` and
`two_view_rotation_covariance_test.py`.

PR stack (2/5 splitting #4823, plus #4912):
1. #4950
2. **#4951** (this PR)
3. #4952
4. #4953
5. #4823
6. #4912

## Summary

Supports whitening relative-rotation residuals in
`CeresRotationAverager` for both fixed-size camera pairs and dynamic
multi-sensor rig blocks:

- Adds `DynamicCovarianceWeightedCostFunction<kNumResiduals>` in
`estimators/cost_functions/utils.h`, the runtime parameter-block
counterpart of `CovarianceWeightedCostFunction` reusing
`internal::ComputeLeftSqrtInformation`.
- Documents the right-perturbation convention of
`RelativeRotationCostFunctor` (`hat_sensor2_from_sensor1 =
sensor2_from_sensor1_prior * Exp(r)`).
- Adds an optional `cam2_from_cam1_cov` argument to
`CeresRotationAverager::AddRelativeRotationResidual` (and
`CeresRotationAverager.add_relative_rotation_residual` in pycolmap) to
whiten individual relative-rotation residuals with
`DynamicCovarianceWeightedCostFunction<3>`.

## Tests

- `DynamicCovarianceWeightedCostFunction.MatchesCovarianceWeighted` and
`CovarianceWeightedCostFunction.RejectsNonPositiveDefiniteCovariance` in
`estimators/cost_functions/utils_test.cc`.
- `CeresRotationAverager.WhitensAddedRelativeRotationResidual` in
`rotation_averaging_ceres_test.cc` (both fixed 2-block frames and
dynamic rig sensors).
- Python binding test in
`src/pycolmap/estimators/rotation_averaging_ceres_test.py`.

PR stack (3/5 splitting #4823, plus #4912):
1. #4950
2. #4951
3. **#4952** (this PR)
4. #4953
5. #4823
6. #4912

)

## Summary

Wires `RotationAveragingReweighting::COVARIANCE` into the global SfM and
rotation-averaging pipelines, and adds CLI flags:

- Adds `MaybeEstimatePoseGraphRotationCovariances` in
`estimators/rotation_averaging.h`: when `reweighting == COVARIANCE`,
estimates the rotation covariances of valid pose graph edges that do not
have one yet (using `ceres.solver_options.num_threads` threads) and
invalidates edges whose covariance cannot be estimated. Throws if
`backend != CERES`, warns and skips when falling back to `L1_IRLS` for
gravity priors, and is a no-op for other reweightings. Also bound in
pycolmap as `maybe_estimate_pose_graph_rotation_covariances`.
- Calls `MaybeEstimatePoseGraphRotationCovariances` in
`GlobalMapper::RotationAveraging`,
`GlobalPipeline::ReconstructMultiComponents`, and
`RotationAveragingPipeline::Run`.
- Registers `--GlobalMapper.ra_ceres_covariance_sigma_floor_deg` and
`--GlobalMapper.ra_ceres_covariance_loss_scale` in `OptionManager`.

The default reweighting remains `UNIFORM` (`COVARIANCE` is opt-in).

## Tests

- `MaybeEstimatePoseGraphRotationCovariances.Nominal` in
`estimators/rotation_averaging_test.cc` and
`test_maybe_estimate_pose_graph_rotation_covariances` in
`pycolmap/estimators/motion_averaging_test.py`.
- `GlobalMapper.WithNoiseAndOutliers`,
`RotationAveragingPipeline.WithNoiseAndOutliers`, and
`GlobalPipeline.MultiComponents` (each also tested with `COVARIANCE`).

---

## Evaluation

### Evaluation Settings
- **Pipeline:**
- SIFT features (`--quality high`, `--use_gpu 1`), exhaustive matching
and `CalibrateViewGraph`.
  - Then `GlobalPipeline` with 1 thread per scene and `random_seed = 0`.
- **ETH3D + ETH3D-Distorted DSLR (50 scenes, 1796 images):**
ground-truth intrinsics priors and ground-truth covisibility filtering.
- **IMC 2023 (7 scenes, 261 images):** uncalibrated cameras, no
covisibility filtering.
- **Metrics:** scene-averaged AUC at `0.5° / 1° / 5° / 10°`, computed
for two stages:
- **RA:** pairwise relative rotation error right after rotation
averaging, before global positioning and BA.
- **E2E:** pairwise relative pose error $`\max(\Delta R, \Delta t)`$
after the full `GlobalPipeline`.
- **Time:** sum of the per-scene, single-threaded runtimes. RA time
covers the pipeline up to rotation averaging, including covariance
estimation.
- **Configurations:**
- `CERES`: the default Ceres configuration (5 Huber warm-start
iterations, then Cauchy with σ = 0.5°).
- `COVARIANCE`: the same configuration with `reweighting = COVARIANCE`,
using the defaults `covariance_loss_scale = 3` and
`covariance_sigma_floor_deg = 0.02°`.
  - The warm-start uses `ceres.loss_function_scale` (0.5° by default).

### Rotation averaging only (RA)

| Method | ETH3D RA AUC | ETH3D time | IMC RA AUC | IMC time |
|---|---|---:|---|---:|
| `L1_IRLS` (default) | 61.43 / 77.28 / 93.43 / 95.65 | 10 s | 1.95 /
6.66 / 26.96 / 39.36 | 1.1 s |
| `CERES` | 78.27 / 87.22 / 95.50 / 96.68 | 11 s | 2.36 / 7.42 / 26.62 /
39.75 | 1.7 s |
| `CERES` + `COVARIANCE` | 84.38 / 90.51 / **96.31 / 97.09** | 29 s |
6.98 / 12.21 / 28.20 / 39.99 | 2.4 s |
| `CERES` + `COVARIANCE`, warm-start σ = 1° (`ceres.loss_function_scale
= 1°`) | 84.32 / 90.50 / **96.31 / 97.09** | 28 s | 7.00 / 12.23 /
**29.22 / 40.92** | 2.4 s |
| `CERES` + `COVARIANCE`, floor 0° | **84.42 / 90.53 / 96.31 / 97.09** |
– | **7.76 / 12.77** / 28.05 / 39.15 | – |
| `CERES` + `COVARIANCE`, floor 0.05°, scale 2 | 84.15 / 90.42 / 96.30 /
97.08 | – | 5.28 / 11.07 / 28.14 / 40.04 | – |
| `CERES` + `COVARIANCE`, inlier-count spanning tree | 84.38 / 90.51 /
96.31 / 97.09 | – | 7.05 / 12.19 / 27.48 / 38.72 | – |
| `CERES` + `COVARIANCE`, warm-start σ = 1°, inlier-count spanning tree
| 84.32 / 90.50 / 96.31 / 97.09 | – | 7.03 / 12.31 / 28.06 / 39.32 | – |

### End-to-end (E2E, seed 0)

| Method | ETH3D E2E AUC | ETH3D time | IMC E2E AUC | IMC time |
|---|---|---:|---|---:|
| `L1_IRLS` (default) | 69.71 / 80.74 / 92.94 / 94.90 | 2759 s | 21.27 /
30.62 / **49.14 / 53.89** | 362 s |
| `CERES` | **69.99 / 81.14 / 93.16 / 95.09** | 2850 s | **24.15 /
33.19** / 45.52 / 48.82 | 584 s |
| `CERES` + `COVARIANCE` | 69.56 / 80.67 / 92.72 / 94.79 | 2903 s |
20.86 / 30.84 / 45.60 / 49.52 | 632 s |
| `CERES` + `COVARIANCE`, warm-start σ = 1° | 69.85 / 80.96 / 93.10 /
95.07 | 2862 s | 20.83 / 30.74 / 46.03 / 50.66 | 617 s |

E2E numbers are measured on the branch merged with the latest `main`.
E2E times come from runs with 12 scenes in parallel and vary with worker
contention: `L1_IRLS` was run separately with fewer concurrent tasks, so
its IMC time is not directly comparable. In both tables, the times of
the `COVARIANCE` rows come from runs before the latest review changes,
which do not change the runtime; the latest runs were slowed down by an
unrelated load on the machine.

On IMC, 3 seeds (mean ± std) give the following avg AUC:

| Method | IMC E2E AUC (3 seeds) |
|---|---|
| `L1_IRLS` (default) | 19.69±2.91 / 29.15±2.85 / **48.24±1.17 /
53.39±0.51** |
| `CERES` | **22.55±2.24 / 32.14±1.34** / 45.04±0.37 / 48.63±0.14 |
| `CERES` + `COVARIANCE` | 18.98±2.42 / 28.81±2.81 / 45.44±0.65 /
49.85±0.30 |
| `CERES` + `COVARIANCE`, warm-start σ = 1° | 18.95±2.43 / 28.60±2.84 /
44.81±1.48 / 49.95±0.79 |

### Takeaways
- Whitening with the estimated covariances greatly improves the
rotations before BA. Avg [email protected]° goes from 78.3 to 84.4 on ETH3D and
from 2.4 to 7.0 on IMC.
- After BA, E2E accuracy is on par on ETH3D: on 49 of 50 scenes,
`COVARIANCE` is +0.03 AUC above `CERES` on average. The average gap
comes from a single 15-image scene, `meadow`, whose global positioning
is bimodal across seeds. On IMC, the differences at the fine thresholds
are within seed noise (2–3 AUC points), and at the coarse thresholds
`COVARIANCE` is slightly above `CERES`.
- The 60° cap on the rotation standard deviation triggers on 1 of the
~16.8k edges of both benchmarks, and the covariance estimation never
fails, so no edge is invalidated for lack of a covariance.
- The overhead comes from estimating the covariances. Single-threaded,
this takes about 0.2 s per ETH3D scene on average. It parallelizes over
edges, and the effect on total E2E time is not measurable. With
`multiple_models`, the covariances are estimated once for the component
split and again in each component mapper; #4912 removes this duplicate.
- Therefore `COVARIANCE` stays opt-in. The default reweighting stays
`UNIFORM`.

<sub>Stack created with <a
href="https://github.com/github/gh-stack">GitHub Stacks CLI</a> • <a
href="https://gh.io/stacks-feedback">Give Feedback 💬</a></sub>

Merge the upstream covariance-weighted rotation averaging and component pose-graph work.

## Change

- Add the upstream covariance-weighted Ceres rotation mode and global-pipeline option. Existing L1_IRLS and UNIFORM defaults remain.
- Reuse the prebuilt pipeline pose graph when constructing component subgraphs.
- Preserve the fork's option registration changes through the disjoint merge.

## Validation

- The object-level merge is conflict-free; incoming paths do not overlap the four existing dirty files.
- Native integration, affected dependent binaries and bindings remain pending. Defaulted C++ arguments do not preserve binary ABI.

## Upstream commits

- d1d92d9 Add COVARIANCE reweighting for Ceres rotation averaging (colmap#4953)
- 761d90d Enable covariance-weighted rotation averaging in global pipelines (colmap#4823)
- fdee2e1 Reuse the pose graph of GlobalPipeline in the component mappers (colmap#4912)

AI disclosure: Totally vibe coded

Refresh the experimental alternative with fork main while retaining its previous head as a parent. The resulting tree is exactly main plus the isolated shared-current-cost-cache deletion: one CUDA file, +1/-25.

## Change

Read the existing cost map directly during the ordered sampled-cost fold. Preserve current plane propagation, source packing, scheduling and fallback mathematics. The cache originates in custom a37c1c3, MVS: Reuse sampled current costs (GPU perf). The earlier alternative is retained through 8481f89; its original experiment was 8b1412f.

Historical Fountain11 photo+geometry at 3072 recorded 76.85 -> 78.82 seconds, one observation per arm, on the older revision. Retain this inconclusive/adverse observation with its earlier revision; current performance has not been measured. No performance experiment was reopened.

## Validation

- Exact candidate CUDA owner compiled with CUDA 13.2 for compute_86/sm_86. In a bounded retained-support context on RTX 3060, both matched targets passed all 13 selected cases: propagation, ordered source folding, checkpoint-HMM and paired-sweep preservation.
- Four complete inputs and four outputs matched across executables: 8,492,128 bytes per arm. Maps were finite with positive depth and checked ordered graph IDs. Complete witnesses use checkpoint_hmm=false; separate checkpoint tests cover both settings without full cross-executable true-mode witnesses.
- Direct owners, link maps and actual loaded providers were checked; managed jobs ended and released their bookings. Later host resize/warp changes have separate owner qualification. This is not a whole-branch build, reconstruction-quality, portability or current-performance result.

## Upstream commits carried forward

- e52aa8e Move pycolmap macOS CI from macos-14 to macos-15 runner (colmap#4933)
- 8007d1a Preserve float weight ordering in maximum spanning trees (colmap#4928)
- dd84102 Bump pypa/gh-action-pypi-publish from 1.14.2 to 1.14.3 in /.github/workflows (colmap#4942)
- 721e185 Add FlexGlue feature matcher (colmap#4908)
- d0a411f FlexGlue: cache the dense features on the GPU (colmap#4915)
- 105817b Hoist camera model dispatch per row in image warps (colmap#4935)
- 27ffad4 Make benchmark translation-angle errors invariant to reconstruction scale (colmap#4932)
- e8f3522 Resolve benchmark paths before changing subprocess directories (colmap#4931)
- f664d5d Clamp visibility coordinates before unsigned conversion (colmap#4943)
- 138119b Keep thin bitmap thumbnails nonempty (colmap#4927)
- b16ed5f Count rig frame covisibility once per 3D point (colmap#4893)
- 93dbbcb Add colmap-review agent skill for code review (colmap#4934)
- 2a6f695 Fix sift_test build on MSVC without the GUI (colmap#4955)
- 78cc729 Fix default prior_position_loss_scale to sqrt(chi2_95_3dof) (colmap#4936)
- 8653c6b Estimate two-view relative rotation covariance (colmap#4950)
- e1317dd FlexGlue: match keypoints in the resized descriptor input frame (colmap#4949)
- 3db6eb0 Extract ConvertPosePriorsToENU and skip pose priors without position (colmap#4937)
- 77cbe48 Store and estimate rotation covariances on PoseGraph (colmap#4951)
- 30a1a81 Support covariance whitening in CeresRotationAverager (colmap#4952)
- e6644e7 Fix -Wunused-result warning in ThreadPool::AddTask (colmap#4958)
- 5b6da88 Clear descriptor-index results for empty queries (colmap#4960)
- d1d92d9 Add COVARIANCE reweighting for Ceres rotation averaging (colmap#4953)
- 761d90d Enable covariance-weighted rotation averaging in global pipelines (colmap#4823)
- fdee2e1 Reuse the pose graph of GlobalPipeline in the component mappers (colmap#4912)

AI disclosure: Totally vibe coded