Repository navigation
Fix distance threshold frame - #16
Merged
Merged
Conversation
fringehush-gif
added a commit
to fringehush-gif/FlexGlue
that referenced
this pull request
Nov 1, 2026
## Summary Measure cycle errors, borders and sparse endpoint distances in each image's descriptor input frame, using both axes instead of a single long-side scale. Returned point coordinates remain in original-image pixels. Derive the frame from existing feature geometry, so cached features and direct ONNX exporter packets need no new metadata. Floating geometry retains fractional borders with integer image sizes and reduced-precision feature caches. FountainP11, 1536 / 3072 px: - Mean wall: eager 1.075 -> 1.076 / 1.631 -> 1.627 s; compiled 0.815 -> 0.826 / 1.402 -> 1.399 s. - Process CPU: eager 1.408 -> 1.404 / 2.460 -> 2.446 s; compiled 1.146 -> 1.158 / 2.244 -> 2.231 s. Peak GPU allocation unchanged. - Eager valid dense samples: 59,250 -> 59,070 / 59,230 -> 59,050. Predicted coordinates and covisibility fields match exactly; this corrects filtering rather than learned predictions. 11 images, 12 pairs, 512-square descriptor, FP32 descriptor/FP16 matcher, 12 CPU threads. Warm each arm/resolution then CTTC, two observations per arm/mode. RSS peaks: eager 2220 -> 2220 / 2294 -> 2285 MiB; compiled 2480 -> 2480 / 2608 -> 2590 MiB. Memory scope is the comparison process with both model instances resident. Inspired by cvg#16; extends the correction to shared exporter/cache inputs and dtype contracts. ## Validation - Analytic cases cover unequal aspect ratios and image scales, square/nonsquare grids, fractional borders, cached integer sizes and float16/32/64 feature geometry. - A 2-pixel descriptor error is rejected at threshold 2 and accepted at 2.1; disabled Python thresholds and runtime tensor thresholds retain their distinct conventions. - Eight fused/unfused ONNX Runtime cases pass with nonsquare exports, dynamic keypoint counts, 1536/3072 image sizes and nonempty accept/all-reject cases. - Linux and Windows CPU suites: 84 tests run each, zero failures/errors (4/5 optional skips). Ten frame/distance tests also pass on CUDA with no skips. Full pretrained ONNX parity is checked by the existing workflow. AI disclosure: totally vibe coded.
sarlinpe
approved these changes
Nov 3, 2026
ahojnnes
pushed a commit
to colmap/colmap
that referenced
this pull request
Nov 3, 2026
## Summary FlexGlue resizes each input image to a fixed descriptor resolution (`512×512` by default), and during training its mutual nearest-neighbor (MNN) search and `max_distance_px` threshold operate in that resized frame. Previously, the exported matcher denormalized predicted correspondences back to each image's original `(W, H)` pixel coordinates and scaled `max_distance_px` by `(max(W0, H0) / 512 + max(W1, H1) / 512) / 2` (see cvg/FlexGlue#16): - On non-square images, Euclidean MNN distances and `max_distance_px` were distorted across axes relative to the `512×512` feature grid (e.g., 50% looser along the short axis of a 3:2 image, and inconsistent on portrait–landscape pairs). - On image pairs with different resolutions, the higher-resolution image dominated the averaged bidirectional distance check. This PR rescales keypoints linearly from `(W, H)` to `(descriptor_width, descriptor_height)` in `KeypointsToTensorData` and passes `descriptor_size` as `image_size0` / `image_size1` to the ONNX matcher: - Keypoint normalization inside the model (`kpts / image_size * 2 - 1`) is unchanged. - MNN search and `max_distance_px` now operate directly in the `512×512` descriptor input frame (`pixel_scale = max(512, 512) / 512 = 1`) using the already-released `v0.1` ONNX weights, and give identical results with models exported both before and after cvg/FlexGlue#16. ## Evaluation End-to-end evaluation with DaD keypoints (2048 and 4096) and default FlexGlue options (`min_covisibility = 0.5`, `early_stop_decay = 0.5`, `filter_threshold = 0.1`), averaged over 3 reconstruction seeds (`mean ± std`), with matching speed measured on an idle A100 GPU: ### ETH3D DSLR (25 scenes, 898 images, 22,569 pairs) | #kpts | Version | [email protected]° | AUC@1° | AUC@5° | AUC@10° | Reg. | Raw matches/pair | Matching (ms/pair) | Verif. (s/scene) | Mapper (s/scene) | |---:|---|---:|---:|---:|---:|---:|---:|---:|---:|---:| | 2048 | `main` | 69.99 ± 0.36 | 82.81 ± 0.31 | 96.06 ± 0.09 | 98.00 ± 0.05 | 898 | 204.9 | 46.0 | 2.42 | 45.7 | | 2048 | **This PR** | **71.02 ± 0.26** | **83.53 ± 0.10** | **96.25 ± 0.02** | **98.10 ± 0.01** | 898 | 199.0 | 45.3 | **1.80 (−26%)** | 44.5 | | 4096 | `main` | 71.56 ± 0.50 | 83.70 ± 0.28 | 96.12 ± 0.04 | 97.99 ± 0.02 | 898 | 408.2 | 47.2 | 4.96 | 84.3 | | 4096 | **This PR** | **71.96 ± 0.77** | **83.96 ± 0.36** | **96.15 ± 0.07** | **98.00 ± 0.04** | 898 | 398.8 | 47.4 | **3.93 (−21%)** | 87.9 | - On ETH3D, verified geometric inliers per pair slightly *increase* (`160.9 → 161.7` at 2k, `309.0 → 312.3` at 4k) while raw matches drop, raising the inlier ratio (`78.5% → 81.2%` at 2k, `75.7% → 78.3%` at 4k) and reducing geometric verification time by 21–26%. ### IMC 2023 (7 scenes, 261 images, 10,935 pairs) | #kpts | Version | [email protected]° | AUC@1° | AUC@5° | AUC@10° | Reg. | Raw matches/pair | Matching (ms/pair) | Verif. (s/scene) | Mapper (s/scene) | |---:|---|---:|---:|---:|---:|---:|---:|---:|---:|---:| | 2048 | `main` | 32.28 ± 1.16 | 54.03 ± 0.88 | 80.76 ± 0.58 | 85.07 ± 0.58 | 255 | 86.2 | 37.0 | 0.94 | 92.2 | | 2048 | **This PR** | 31.69 ± 0.37 | 53.39 ± 0.32 | 80.65 ± 0.22 | 85.05 ± 0.27 | 255 | 80.6 | 37.0 | **0.72 (−23%)** | **85.7 (−7%)** | | 4096 | `main` | 31.82 ± 0.47 | 54.26 ± 0.51 | 82.85 ± 0.78 | 87.58 ± 1.22 | 259 | 191.2 | 40.1 | 2.37 | 202.4 | | 4096 | **This PR** | 27.67 ± 1.72 | 50.01 ± 2.09 | 81.34 ± 0.15 | 86.87 ± 0.77 | **260** | 180.9 | 40.1 | **1.91 (−19%)** | **186.3 (−8%)** | - Mixed portrait/landscape scenes improve (`heritage/dioscuri`: `+2.96` [email protected]° at 2k, `+1.10` [email protected]° / `+1.97` AUC@5° at 4k), and geometric verification is 19–23% faster. - On the 7-scene IMC average, the drop in fine-threshold AUC at 4k comes primarily from `heritage/wall` (34 images, `6048×4032`), where registering all 34/34 views across all 3 seeds (vs. 2/3 seeds on `main`) raises `wall` `AUC@10°` (`+1.74`) while 4 marginal wide-baseline views are registered with coarse poses that lower `wall` `[email protected]°`, alongside tightening the short-axis MNN window on 3:2 images from `4.5` to `3.0` resized pixels (`heritage/cyprus` `24 → 23` registered at 4k). ## Tests - Extended `FlexGlueTest.MatchTranslatedImages` to verify that anisotropically rescaling an image's camera dimensions and keypoints (`4W × 2H`) leaves the matches unchanged.
fringehush-gif
referenced
this pull request
in fringehush-gif/FlexGlue
Nov 9, 2026
## Summary Reuse caller-provided refinement features and update the notebook's RaCo source. Refresh from official c6cbc7a while retaining the fork's tested defaults. ## Changes - Adopt cfd752c, "Accept precomputed refinement features" (#10), and 7b0b5f3, "load RaCo from main" (#17). - Retain our descriptor-frame conversion and empty-input handling when resolving c6cbc7a, "Fix distance threshold frame" (#16). - Defer 6075232, "Compile image encoding and RoMaV2 refinement" (#9). Earlier encoder tests had mixed end-to-end cost/output results; compiled refiner output lifetime needs a replay test. Existing compilation stays intact. ## Validation - Linux Torch 2.14.1: 116 tests run, zero failures/errors; 11 CUDA tests and the optional ONNX test class skipped. - Added both-side and mixed cached/missing refinement-routing regressions. - Independent review accepted the narrowed source; no new performance claim. AI disclosure: Totally vibe coded
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
So far the distance thresholds were rescaled to 512 / max(h,w). This is inconsistent with how we trained it for: There we apply thresholding in the actual input shape of desriptors, so (512,512). This PR applies this fix. This boosts results on some benchmarks.