Skip to content

Clear descriptor-index results for empty queries (#4960) - #1

Merged
ak-constructn merged 12 commits into
constructn-corp:mainfrom
colmap:main
Nov 10, 2026
Merged

ak-constructn merged 12 commits into
constructn-corp:mainfrom
colmap:main

Conversation

@satishontrack

Copy link
Copy Markdown

FeatureDescriptorIndex::Search returns early for an empty descriptor matrix without updating its output matrices. A C++ caller that reuses those matrices after a nonempty query receives the previous query's indices and distances for an empty input.

Clear both matrices in the existing empty-query branch, matching the existing empty-index result convention. Type and descriptor-width validation and nonempty FAISS searches stay unchanged. The current SIFT pipeline already filters empty images before this call; this fix covers direct use of the public index utility.

I extended the existing parameterized native test to search an empty matrix after filling both outputs. All four combinations fail the new assertions on pristine production and pass with the fix, covering SIFT/ALIKED and flat/IVF indexes. All five module tests pass. A separate native check covers 72 empty queries and 312 ordinary, repeated and rebuild controls; ordinary results match pristine output exactly. Clang-format, scoped Clang-Tidy and whitespace checks pass.

I validated the actual C++ owner with MSVC Release and generic CPU FAISS 1.15.1. I did not run a full application build, reconstruction datasets, GPU builds, dynamic-dispatch builds or other operating systems.

AI assistance disclosure: AI-assisted development tools were used during investigation and implementation. The reported tests were run against the final diff.

aswanth-07 and others added 12 commits November 6, 2026 08:09
`FeatureDescriptorIndex::Search` returns early for an empty descriptor
matrix without updating its output matrices. A C++ caller that reuses
those matrices after a nonempty query receives the previous query's
indices and distances for an empty input.

Clear both matrices in the existing empty-query branch, matching the
existing empty-index result convention. Type and descriptor-width
validation and nonempty FAISS searches stay unchanged. The current SIFT
pipeline already filters empty images before this call; this fix covers
direct use of the public index utility.

I extended the existing parameterized native test to search an empty
matrix after filling both outputs. All four combinations fail the new
assertions on pristine production and pass with the fix, covering
SIFT/ALIKED and flat/IVF indexes. All five module tests pass. A separate
native check covers 72 empty queries and 312 ordinary, repeated and
rebuild controls; ordinary results match pristine output exactly.
Clang-format, scoped Clang-Tidy and whitespace checks pass.

I validated the actual C++ owner with MSVC Release and generic CPU FAISS
1.15.1. I did not run a full application build, reconstruction datasets,
GPU builds, dynamic-dispatch builds or other operating systems.

AI assistance disclosure: AI-assisted development tools were used during
investigation and implementation. The reported tests were run against
the final diff.
## Summary

Adds `RotationAveragingReweighting::COVARIANCE` to whiten
relative-rotation constraints in the Ceres rotation averaging backend
using per-edge rotation covariances:

- `CeresRotationAverager` whitens each residual, $`r_{ij} \mapsto
\Sigma_{R,ij}^{-1/2} r_{ij}`$, where $`\Sigma_{R,ij}`$ is regularized
with an isotropic floor, $`\Sigma_{R,ij} + \sigma_{\text{floor}}^2 I_3`$
(`CeresRotationAveragerOptions::covariance_sigma_floor_deg`, 0.02° by
default). The pose graph keeps the raw covariances.
- The robust loss acts on the whitened, unitless residual with scale
`CeresRotationAveragerOptions::covariance_loss_scale` (3.0 standard
deviations by default), in place of `loss_function_scale` (in degrees).
- Splits Ceres rotation averaging preparation into `CheckPoseGraph`,
`PrepareWeights`, and `InitializeRotations` (reusing
`internal::TryComputeLeftSqrtInformation` and
`DynamicCovarianceWeightedCostFunction::FromLeftSqrtInformation`) so
both `RotationEstimator` and `CeresRotationAverager` validate
connectivity, sensor rotations, loss scales,
`covariance_sigma_floor_deg`, and that every valid edge has a finite,
symmetric positive-definite floored covariance before MST initialization
modifies the reconstruction, without repeating connected-components or
per-edge Cholesky factorizations.
- Adds `use_rotation_covariance` to `InitializeFromMaximumSpanningTree`,
preferring edges with the smallest $`\mathrm{tr}(\Sigma_R)`$ via weights
$`-\log \mathrm{tr}(\Sigma_R)`$. `RotationEstimator` runs MST
initialization once before the warm-start.
- Keeps the Huber warm-start on angular residuals with `UNIFORM` weights
so confidently wrong relative rotations (e.g., from symmetric
structures) do not dominate early iterations.
- `COVARIANCE` is unsupported by `L1_IRLS` (throws), and is ignored with
a warning when `CERES` falls back to `L1_IRLS` for gravity priors.
- Binds `RotationAveragingReweighting.COVARIANCE`,
`covariance_loss_scale`, and `covariance_sigma_floor_deg` in pycolmap.

## Tests

- `CeresRotationAverager.CovarianceWhitening`,
`CovarianceDownweightsUncertainEdges`, `CovarianceReweightingWithRigs`,
and `RotationEstimator.CeresChecksInputsBeforeInitialization` in
`rotation_averaging_ceres_test.cc`.
- `RotationAveraging.InitializeFromMaximumSpanningTreeWithCovariance`
and `RotationAveraging.CovarianceReweighting` in
`rotation_averaging_test.cc`.
- Python binding tests in `motion_averaging_test.py` and
`rotation_averaging_ceres_test.py`.

<sub>Stack created with <a
href="https://github.com/github/gh-stack">GitHub Stacks CLI</a> • <a
href="https://gh.io/stacks-feedback">Give Feedback 💬</a></sub>
)

## Summary

Wires `RotationAveragingReweighting::COVARIANCE` into the global SfM and
rotation-averaging pipelines, and adds CLI flags:

- Adds `MaybeEstimatePoseGraphRotationCovariances` in
`estimators/rotation_averaging.h`: when `reweighting == COVARIANCE`,
estimates the rotation covariances of valid pose graph edges that do not
have one yet (using `ceres.solver_options.num_threads` threads) and
invalidates edges whose covariance cannot be estimated. Throws if
`backend != CERES`, warns and skips when falling back to `L1_IRLS` for
gravity priors, and is a no-op for other reweightings. Also bound in
pycolmap as `maybe_estimate_pose_graph_rotation_covariances`.
- Calls `MaybeEstimatePoseGraphRotationCovariances` in
`GlobalMapper::RotationAveraging`,
`GlobalPipeline::ReconstructMultiComponents`, and
`RotationAveragingPipeline::Run`.
- Registers `--GlobalMapper.ra_ceres_covariance_sigma_floor_deg` and
`--GlobalMapper.ra_ceres_covariance_loss_scale` in `OptionManager`.

The default reweighting remains `UNIFORM` (`COVARIANCE` is opt-in).

## Tests

- `MaybeEstimatePoseGraphRotationCovariances.Nominal` in
`estimators/rotation_averaging_test.cc` and
`test_maybe_estimate_pose_graph_rotation_covariances` in
`pycolmap/estimators/motion_averaging_test.py`.
- `GlobalMapper.WithNoiseAndOutliers`,
`RotationAveragingPipeline.WithNoiseAndOutliers`, and
`GlobalPipeline.MultiComponents` (each also tested with `COVARIANCE`).

---

## Evaluation

### Evaluation Settings
- **Pipeline:**
- SIFT features (`--quality high`, `--use_gpu 1`), exhaustive matching
and `CalibrateViewGraph`.
  - Then `GlobalPipeline` with 1 thread per scene and `random_seed = 0`.
- **ETH3D + ETH3D-Distorted DSLR (50 scenes, 1796 images):**
ground-truth intrinsics priors and ground-truth covisibility filtering.
- **IMC 2023 (7 scenes, 261 images):** uncalibrated cameras, no
covisibility filtering.
- **Metrics:** scene-averaged AUC at `0.5° / 1° / 5° / 10°`, computed
for two stages:
- **RA:** pairwise relative rotation error right after rotation
averaging, before global positioning and BA.
- **E2E:** pairwise relative pose error $`\max(\Delta R, \Delta t)`$
after the full `GlobalPipeline`.
- **Time:** sum of the per-scene, single-threaded runtimes. RA time
covers the pipeline up to rotation averaging, including covariance
estimation.
- **Configurations:**
- `CERES`: the default Ceres configuration (5 Huber warm-start
iterations, then Cauchy with σ = 0.5°).
- `COVARIANCE`: the same configuration with `reweighting = COVARIANCE`,
using the defaults `covariance_loss_scale = 3` and
`covariance_sigma_floor_deg = 0.02°`.
  - The warm-start uses `ceres.loss_function_scale` (0.5° by default).

### Rotation averaging only (RA)

| Method | ETH3D RA AUC | ETH3D time | IMC RA AUC | IMC time |
|---|---|---:|---|---:|
| `L1_IRLS` (default) | 61.43 / 77.28 / 93.43 / 95.65 | 10 s | 1.95 /
6.66 / 26.96 / 39.36 | 1.1 s |
| `CERES` | 78.27 / 87.22 / 95.50 / 96.68 | 11 s | 2.36 / 7.42 / 26.62 /
39.75 | 1.7 s |
| `CERES` + `COVARIANCE` | 84.38 / 90.51 / **96.31 / 97.09** | 29 s |
6.98 / 12.21 / 28.20 / 39.99 | 2.4 s |
| `CERES` + `COVARIANCE`, warm-start σ = 1° (`ceres.loss_function_scale
= 1°`) | 84.32 / 90.50 / **96.31 / 97.09** | 28 s | 7.00 / 12.23 /
**29.22 / 40.92** | 2.4 s |
| `CERES` + `COVARIANCE`, floor 0° | **84.42 / 90.53 / 96.31 / 97.09** |
– | **7.76 / 12.77** / 28.05 / 39.15 | – |
| `CERES` + `COVARIANCE`, floor 0.05°, scale 2 | 84.15 / 90.42 / 96.30 /
97.08 | – | 5.28 / 11.07 / 28.14 / 40.04 | – |
| `CERES` + `COVARIANCE`, inlier-count spanning tree | 84.38 / 90.51 /
96.31 / 97.09 | – | 7.05 / 12.19 / 27.48 / 38.72 | – |
| `CERES` + `COVARIANCE`, warm-start σ = 1°, inlier-count spanning tree
| 84.32 / 90.50 / 96.31 / 97.09 | – | 7.03 / 12.31 / 28.06 / 39.32 | – |

### End-to-end (E2E, seed 0)

| Method | ETH3D E2E AUC | ETH3D time | IMC E2E AUC | IMC time |
|---|---|---:|---|---:|
| `L1_IRLS` (default) | 69.71 / 80.74 / 92.94 / 94.90 | 2759 s | 21.27 /
30.62 / **49.14 / 53.89** | 362 s |
| `CERES` | **69.99 / 81.14 / 93.16 / 95.09** | 2850 s | **24.15 /
33.19** / 45.52 / 48.82 | 584 s |
| `CERES` + `COVARIANCE` | 69.56 / 80.67 / 92.72 / 94.79 | 2903 s |
20.86 / 30.84 / 45.60 / 49.52 | 632 s |
| `CERES` + `COVARIANCE`, warm-start σ = 1° | 69.85 / 80.96 / 93.10 /
95.07 | 2862 s | 20.83 / 30.74 / 46.03 / 50.66 | 617 s |

E2E numbers are measured on the branch merged with the latest `main`.
E2E times come from runs with 12 scenes in parallel and vary with worker
contention: `L1_IRLS` was run separately with fewer concurrent tasks, so
its IMC time is not directly comparable. In both tables, the times of
the `COVARIANCE` rows come from runs before the latest review changes,
which do not change the runtime; the latest runs were slowed down by an
unrelated load on the machine.

On IMC, 3 seeds (mean ± std) give the following avg AUC:

| Method | IMC E2E AUC (3 seeds) |
|---|---|
| `L1_IRLS` (default) | 19.69±2.91 / 29.15±2.85 / **48.24±1.17 /
53.39±0.51** |
| `CERES` | **22.55±2.24 / 32.14±1.34** / 45.04±0.37 / 48.63±0.14 |
| `CERES` + `COVARIANCE` | 18.98±2.42 / 28.81±2.81 / 45.44±0.65 /
49.85±0.30 |
| `CERES` + `COVARIANCE`, warm-start σ = 1° | 18.95±2.43 / 28.60±2.84 /
44.81±1.48 / 49.95±0.79 |

### Takeaways
- Whitening with the estimated covariances greatly improves the
rotations before BA. Avg [email protected]° goes from 78.3 to 84.4 on ETH3D and
from 2.4 to 7.0 on IMC.
- After BA, E2E accuracy is on par on ETH3D: on 49 of 50 scenes,
`COVARIANCE` is +0.03 AUC above `CERES` on average. The average gap
comes from a single 15-image scene, `meadow`, whose global positioning
is bimodal across seeds. On IMC, the differences at the fine thresholds
are within seed noise (2–3 AUC points), and at the coarse thresholds
`COVARIANCE` is slightly above `CERES`.
- The 60° cap on the rotation standard deviation triggers on 1 of the
~16.8k edges of both benchmarks, and the covariance estimation never
fails, so no edge is invalidated for lack of a covariance.
- The overhead comes from estimating the covariances. Single-threaded,
this takes about 0.2 s per ETH3D scene on average. It parallelizes over
edges, and the effect on total E2E time is not measurable. With
`multiple_models`, the covariances are estimated once for the component
split and again in each component mapper; #4912 removes this duplicate.
- Therefore `COVARIANCE` stays opt-in. The default reweighting stays
`UNIFORM`.

<sub>Stack created with <a
href="https://github.com/github/gh-stack">GitHub Stacks CLI</a> • <a
href="https://gh.io/stacks-feedback">Give Feedback 💬</a></sub>
## Summary

With `multiple_models` (the default), `GlobalPipeline` loads the full
pose graph, splits it into components with rotation averaging, and then
runs a `GlobalMapper` per component. Each component mapper reloaded its
pose graph from the component database cache, so all edge data computed
by the pipeline was lost. With `COVARIANCE` reweighting, the rotation
covariances were thus estimated twice: once for the component split, and
once again in each component mapper.

This PR passes each component mapper the subgraph of the pipeline's pose
graph instead:

- New `PoseGraph::Subgraph(image_ids)` returns the edges between the
given images, copied whole (relative pose, rotation covariance, match
count, validity, prior flag), so that edge data added in the future
carries over too. Also exposed in pycolmap as `PoseGraph.subgraph`.
- `GlobalMapper::BeginReconstruction` optionally takes a pose graph, and
uses its subgraph between the images of its database cache. Without one,
it loads the pose graph from the database cache as before.
- `GlobalPipeline` passes its full pose graph to each component mapper.
The component cache also includes the other images of the component's
frames (multi-camera rigs), so their edges carry over too. The mapper
then skips covariance estimation, since every edge already has one.

## Tests

- `PoseGraph.Subgraph`: edge selection and whole-edge copy, including
validity.
- `PoseGraph.SubgraphMatchesComponentDatabaseCache`: on a two-camera rig
dataset, the subgraph of a component has exactly the same edges and
relative poses as the pose graph loaded from
`DatabaseCache::CreateFromCache`. This guards the equivalence the
pipeline relies on.
- `GlobalMapper.WithGivenPoseGraph`: the mapper uses the given pose
graph, and drops its edges to images outside the database cache.
- `test_pose_graph_subgraph` for the binding.

## Evaluation

Covariance estimation now runs once per scene instead of once plus once
per component, which saves about 0.2 s per ETH3D scene single-threaded.

Results are unchanged up to floating-point rounding. The subgraph's hash
map iterates its edges in a different order than a pose graph loaded
from the component cache, which changes the order of floating-point
sums. This also affects the default `UNIFORM` reweighting, in scenes
split into several components:
- RA-only per-scene AUCs on ETH3D and IMC 2023 differ by less than
1e-12.
- E2E averages are identical. Per scene, only IMC `dioscuri` (a scene
that already amplifies tiny perturbations, −0.06 to +0.32 AUC across
seeds) and ETH3D `courtyard` (+0.006 AUC) change.

<sub>Stack created with <a
href="https://github.com/github/gh-stack">GitHub Stacks CLI</a> • <a
href="https://gh.io/stacks-feedback">Give Feedback 💬</a></sub>
The tutorial names its output `dense.ply`, but the Python binding
defaults to `output_type="bin"` and passes that path to the binary
reconstruction directory writer. Pass `output_type="ply"` so readers get
the point cloud shown in the example.

Tested both calls with PyCOLMAP 4.2.1 on a synthetic PINHOLE
reconstruction and an empty fusion configuration. The original call
raises the directory error reported in #4280. The updated call writes a
file with a PLY header. This addresses that exception independently of
how many points a dataset fuses.

Refs #4280.
GTEST_FAIL() is a fatal failure, so the break statements after it in the
default branches of the Merge test switches are unreachable. Remove both
of them.
`RANSAC::Report::num_trials` currently includes reservations that fail
the trial-budget or abort check before sampling. For example, a
one-trial budget reports two iterations with one thread and five with
four threads. A zero-trial budget also reports work that never ran.
`LORANSAC` shares the same reporting error.

I count the iterations that pass both checks with an OpenMP reduction.
The reservation counter, sampling, stopping conditions and model
selection remain unchanged. This also gives accurate trial totals to the
existing fundamental-matrix benchmark.

The two regression tests cover zero, one and five trials with one and
four threads. All 12 count assertions fail on pristine production; all
12 tests across both native modules pass with the fix, both with OpenMP
and with it disabled. A separate 900-call counting matrix confirms that
each report equals the actual minimal-estimator calls. All 108 seeded
similarity-transform model, mask and support controls match pristine
output exactly.

I validated MSVC Release on Windows. Six alternating timing rounds show
median changes from -3.4% to +5.8% across the serial paths, including a
+4.4 us/call no-OpenMP LORANSAC diagnostic. The per-round timings are
noisy; this is a reporting correction without a speed claim.
Changed-file formatting, scoped Clang-Tidy and whitespace checks pass.
Full application, reconstruction-dataset, GPU and other-platform
validation remain unrun.

AI assistance disclosure: AI-assisted development tools were used during
investigation and implementation. The reported tests were run against
the final diff.

---------

Co-authored-by: Johannes Schönberger <[email protected]>
The dataset downloader imports `urllib2` and `urlparse`, so loading the
script on Python 3 raises `ModuleNotFoundError` before the CLI can parse
any options. Use the standard `urllib.request` and `urllib.parse`
modules and close the HTTP response after parsing the XML page.

The regression test reads a Flickr-shaped XML response and checks the
image URL and local filename passed to the retriever, including a URL
with a query string. Network access is mocked. Verified `python
python/util/flickr_downloader.py --help` and `pytest
python/util/flickr_downloader_test.py` on Python 3.12. Ruff lint and
format checks pass.

---------

Co-authored-by: Johannes Schönberger <[email protected]>
After extracting the competition archive, `download_imc2024` joins the
dataset path to `category_path` a second time. With `--data_path data`,
the destination becomes `data/imc2024/data/imc2024/train/all` and moving
the extracted scene fails. Move each scene directly to `category_path`
and identify the `all` directory by its path component on every
platform.

`pytest benchmark/reconstruction/download_test.py` passes on Python
3.12. The test extracts a small local ZIP, stubs only the Kaggle
download command, and verifies the image bytes at
`data/imc2024/train/all/scene/images/one.jpg`. Ruff lint and format
checks pass.

---------

Co-authored-by: Johannes Schönberger <[email protected]>
…4957)

## Summary

With a single matching worker per GPU, FlexGlue leaves the GPU
underutilized while the CPU decodes high-resolution images, resizes
them, prepares ONNX tensors, and processes early-stopping `If` checks.
Spawning separate matchers per worker would duplicate both ONNX
Runtime's per-session GPU arena (~2.2–2.8 GiB) and the 512 MiB GPU
dense-feature cache.

Because `Ort::Session::Run` supports concurrent calls across threads,
multiple worker threads per CUDA GPU can share a single
`FlexGlueFeatureMatcher` instance:
- Switch `FlexGlueFeatureMatcher::dense_cache_` from `LRUCache` to
`ThreadSafeLRUCache`, which also deduplicates concurrent loads when
multiple workers request the same uncached image simultaneously.
- Make `DenseFeatures::checked_camera_size` `std::atomic<bool>` so
`ImageSize()` is lock-free and race-free across workers.
- Add `SharedFeatureMatcher` in `feature_matching_utils` so one
designated creator worker initializes the matcher (propagating
initialization errors once via `Setup()`) while sibling workers wait for
and share the same instance.
- Add `FlexGlueMatchingOptions::num_workers_per_gpu = 3` (exposed on the
CLI as `--FlexGlueMatching.num_workers_per_gpu`, in `pycolmap`, and
documented in `doc/features.rst`).

Matches are bit-exact with `num_workers_per_gpu = 1`; lowering
`num_workers_per_gpu` reduces peak GPU arena memory from concurrent
in-flight transformer activations.

## Evaluation

Evaluated on 3 ETH3D DSLR scenes (`delivery_area`, `terrains`, `office`:
112 images at 24–25.7 MP, 2,132 exhaustive pairs) with DaD keypoints
(2048 and 4096) and default FlexGlue options on an idle NVIDIA
A100-SXM4-40GB GPU (12 vCPUs):

| #kpts | `num_workers_per_gpu` | Matching (ms/pair) | Speedup | Active
GPU Util | Peak VRAM (MiB) | Raw matches |
|---:|---:|---:|---:|---:|---:|---:|
| 2048 | `1` (`main`) | 51.95 | 1.00× | 56.9% | 2,659 | 434,456 |
| 2048 | `2` | 37.47 | 1.39× | 66.4% | 3,793 | 434,456 |
| 2048 | **`3` (default)** | **33.32** | **1.56× (−36%)** | **70.1%** |
4,937–5,905 | **434,456** |
| 2048 | `4` | 31.94 | 1.63× | 73.7% | 5,033 | 434,456 |
| 4096 | `1` (`main`) | 54.93 | 1.00× | 58.2% | 3,675 | 868,694 |
| 4096 | `2` | 39.65 | 1.39× | 68.3% | 4,817 | 868,694 |
| 4096 | **`3` (default)** | **35.71** | **1.54× (−35%)** | **72.3%** |
5,957 | **868,694** |

## Tests

- `FlexGlueTest.ConcurrentMatchThreadSafety`: verifies concurrent
`Match()` calls across 3 threads hitting cold and warm
`ThreadSafeLRUCache` entries return identical matches.
- `FeatureMatcherController.SharedFeatureMatcherWorkersMatchPairs` and
`SetupPropagatesMatcherCreationErrors`: verify shared matcher execution
across 3 workers and single-rethrow error propagation when matcher
creation fails.
- `FlexGlueMatchingOptions.Check`: verifies `num_workers_per_gpu`
default (`3`) and bounds check (`> 0`).
Stacked on #4957.

### Summary
Decouples CPU image decoding/preprocessing parallelism
(`FlexGlueMatchingOptions::num_workers_per_gpu`, $N$) from concurrent
GPU model execution (`FlexGlueMatchingOptions::num_inferences_per_gpu`,
$M$) in `FlexGlue`:

- **`ONNXInferenceLimiter` (`util/onnx.{h,cc}`)**: Slot-assigning
counting semaphore bounding concurrent GPU inferences across a larger
pool of worker threads, paired with `OrtCUDAProviderOptionsV2`
(`arena_extend_strategy=kSameAsRequested`,
`use_ep_level_unified_stream=1`) and lazy per-slot CUDA `Ort::Session`
replicas (`allow_session_replicas=false` on `FlexGlue`'s 344 MB dense
descriptor so its weights are never duplicated).
- **`FlexGlueMatchingOptions::num_inferences_per_gpu`
(`feature/flexglue.{h,cc}`, CLI, `pycolmap`, docs)**: Bounds concurrent
`descriptor_` + `matcher_` executions per CUDA GPU (default
`num_workers_per_gpu = 4`, `num_inferences_per_gpu = 2`), and releases
decoded full-resolution `Bitmap` buffers immediately after
`ResizeBilinearToCHW` so workers waiting on the semaphore hold only the
resized `512x512` input tensor (`~3 MB` instead of `72–150 MB`).

### Benchmark: $(N, M)$ Grid Search on `delivery_area` (44 images @ 24
MP, 946 pairs, NVIDIA A100-SXM4-40GB)

#### `FLEXGLUE` (`K=2048` keypoints)

| $N$ (`workers`) | $M$ (`inferences`) | Block ms/pair | Block speedup |
Total ms/pair | GPU util | Mem BW | Mean / Peak Power | Peak VRAM |
Matches |
|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| `1` | `1` | `42.94` | `1.00x` | `46.74` | `56.1%` | `9.5%` | `186 /
304 W` | `1,964 MiB` | `169,581` |
| `3` | `1` | `32.99` | `1.30x` | `36.85` | `70.0%` | `12.2%` | `226 /
310 W` | `1,960 MiB` | `169,581` |
| `6` | `1` | `30.84` | `1.39x` | `34.63` | `74.6%` | `13.0%` | `239 /
311 W` | `1,960 MiB` | `169,581` |
| `2` | `2` | `29.61` | `1.45x` | `33.36` | `63.4%` | `17.3%` | `249 /
398 W` | `2,788 MiB` | `169,581` |
| `4` | `2` | `25.46` | `1.69x` | `29.24` | `71.6%` | `19.7%` | `286 /
407 W` | `2,788 MiB` | `169,581` |
| `3` | `3` | `26.86` | `1.60x` | `30.57` | `66.4%` | `22.2%` | `272 /
421 W` | `3,600 MiB` | `169,581` |
| `6` | `3` | `24.40` | `1.76x` | `28.22` | `71.4%` | `24.2%` | `289 /
417 W` | `3,610 MiB` | `169,581` |
| `8` | `3` | **`23.67`** | **`1.81x`** | **`27.40`** | `73.6%` |
`24.7%` | `296 / 419 W` | `3,616 MiB` | `169,581` |
| `4` | `4` | `26.34` | `1.63x` | `30.13` | `67.0%` | `25.0%` | `276 /
415 W` | `4,412 MiB` | `169,581` |
| `8` | `4` | `24.17` | `1.78x` | `27.97` | `72.8%` | `26.3%` | `291 /
414 W` | `4,440 MiB` | `169,581` |
| `6` | `6` | `27.09` | `1.59x` | `30.89` | `66.9%` | `25.8%` | `272 /
416 W` | `6,038 MiB` | `169,581` |
| `8` | `8` | `27.89` | `1.54x` | `31.71` | `66.6%` | `24.5%` | `266 /
422 W` | `7,708 MiB` | `169,581` |
| `12` | `12` | `29.23` | `1.47x` | `33.04` | `65.6%` | `20.6%` | `253 /
417 W` | `10,982 MiB` | `169,581` |
## Summary

Update the visibility pyramid only when an image feature gains its first
triangulated correspondence or loses its last. Extra supporting
correspondences leave occupied cells unchanged, so repeating every
pyramid-level update does unnecessary work.

This is a small inherited optimization, based directly on official
COLMAP 3c1a1b9.

## Changes

- Move SetPoint/ResetPoint into the existing zero/one visibility
transitions.
- Extend the maintained score test with repeated support, distinct
colliding features, partial/final removal and underflow checks.
- Extend AddImage with observation removal and reconstruction of the
manager, avoiding unrelated fork registration behavior.

## Background

The update pattern dates to fe8f82a,
"Initial commit" (2016), in src/base/image.cc. It was retained by
eaebe3c, "Add ObservationManager
(#2575)".

## Results

The source-equivalent transition change was measured in a composed-fork
fixed bookkeeping probe: Add64/Delete32/Delete32 on 16 images, 64 tracks
and 120 pairs.

| Platform | Before | After |
| --- | ---: | ---: |
| Linux/WSL | 1.395 ms | 0.640 ms |
| Windows | 2.776 ms | 1.438 ms |

Each table value is the arithmetic mean of four fresh process means per
arm (ABBAABBA). Each process mean averages ten measured batches after
two warmups. Canonical inputs and outputs agree within each platform.
These are fixed bookkeeping timings, not reconstruction throughput
measurements.

## Validation

- Independent source review accepted the clean-base production change
and adapted lifecycle fixture.
- Clean base and candidate each passed all 15 maintained manager cases,
without skips.
- Configured clang-tidy-18 checked observation_manager.cc in both arms
and the maintained test with no displayed source diagnostics;
generated/header warnings were suppressed.
- Reused retained pristine-upstream support and rebuilt the two selected
consumers of newer RANSAC headers; final source, link and provider
checks passed.
- Separately, the composed-fork version passed 18 manager and two mapper
cases on Linux and Windows.

AI disclosure: Totally vibe coded
@ak-constructn
ak-constructn merged commit d0a3a01 into constructn-corp:main Nov 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants