Repository navigation
FlexGlue: match keypoints in the resized descriptor input frame - #4949
Merged
Merged
Conversation
Rescale keypoints linearly from the original image dimensions to the resized descriptor input dimensions (512x512 by default) and pass that size to the ONNX matcher, so that mutual nearest-neighbor search and max_distance_px operate in the descriptor input frame (matching cvg/FlexGlue#16, with models exported both before and after it).
ahojnnes
approved these changes
Nov 3, 2026
fringehush-gif
added a commit
to fringehush-gif/colmap
that referenced
this pull request
Nov 3, 2026
## Change Import relative-rotation covariance estimation and the corrected resized input frame for FlexGlue keypoints. The merge is clean; retain the fork's default-off optimistic scoring and both triangulation and covariance test registrations. ## Upstream commits - `8653c6bd14d159f4f8bdf99bb220a7f54d4101b4` Estimate two-view relative rotation covariance (colmap#4950) - `e1317ddd0e23abf70e2ad26e58139afb55bf9eac` FlexGlue: match keypoints in the resized descriptor input frame (colmap#4949) ## Validation Private integration preview and source checks passed. Native C++ and Python qualification remain pending; no covariance calibration or pipeline claim. AI-assisted integration.
fringehush-gif
added a commit
to fringehush-gif/colmap
that referenced
this pull request
Nov 4, 2026
## Change Import relative-rotation covariance estimation and the corrected resized input frame for FlexGlue keypoints. The merge is clean; retain the fork's default-off optimistic scoring and both triangulation and covariance test registrations. ## Upstream commits - `8653c6bd14d159f4f8bdf99bb220a7f54d4101b4` Estimate two-view relative rotation covariance (colmap#4950) - `e1317ddd0e23abf70e2ad26e58139afb55bf9eac` FlexGlue: match keypoints in the resized descriptor input frame (colmap#4949) ## Validation Private integration preview and source checks passed. Native C++ and Python qualification remain pending; no covariance calibration or pipeline claim. AI-assisted integration.
fringehush-gif
added a commit
to fringehush-gif/colmap
that referenced
this pull request
Nov 7, 2026
Refresh the experimental alternative with fork main while retaining its previous head as a parent. The resulting tree is exactly main plus the isolated shared-current-cost-cache deletion: one CUDA file, +1/-25. ## Change Read the existing cost map directly during the ordered sampled-cost fold. Preserve current plane propagation, source packing, scheduling and fallback mathematics. The cache originates in custom a37c1c3, MVS: Reuse sampled current costs (GPU perf). The earlier alternative is retained through 8481f89; its original experiment was 8b1412f. Historical Fountain11 photo+geometry at 3072 recorded 76.85 -> 78.82 seconds, one observation per arm, on the older revision. Retain this inconclusive/adverse observation with its earlier revision; current performance has not been measured. No performance experiment was reopened. ## Validation - Exact candidate CUDA owner compiled with CUDA 13.2 for compute_86/sm_86. In a bounded retained-support context on RTX 3060, both matched targets passed all 13 selected cases: propagation, ordered source folding, checkpoint-HMM and paired-sweep preservation. - Four complete inputs and four outputs matched across executables: 8,492,128 bytes per arm. Maps were finite with positive depth and checked ordered graph IDs. Complete witnesses use checkpoint_hmm=false; separate checkpoint tests cover both settings without full cross-executable true-mode witnesses. - Direct owners, link maps and actual loaded providers were checked; managed jobs ended and released their bookings. Later host resize/warp changes have separate owner qualification. This is not a whole-branch build, reconstruction-quality, portability or current-performance result. ## Upstream commits carried forward - e52aa8e Move pycolmap macOS CI from macos-14 to macos-15 runner (colmap#4933) - 8007d1a Preserve float weight ordering in maximum spanning trees (colmap#4928) - dd84102 Bump pypa/gh-action-pypi-publish from 1.14.2 to 1.14.3 in /.github/workflows (colmap#4942) - 721e185 Add FlexGlue feature matcher (colmap#4908) - d0a411f FlexGlue: cache the dense features on the GPU (colmap#4915) - 105817b Hoist camera model dispatch per row in image warps (colmap#4935) - 27ffad4 Make benchmark translation-angle errors invariant to reconstruction scale (colmap#4932) - e8f3522 Resolve benchmark paths before changing subprocess directories (colmap#4931) - f664d5d Clamp visibility coordinates before unsigned conversion (colmap#4943) - 138119b Keep thin bitmap thumbnails nonempty (colmap#4927) - b16ed5f Count rig frame covisibility once per 3D point (colmap#4893) - 93dbbcb Add colmap-review agent skill for code review (colmap#4934) - 2a6f695 Fix sift_test build on MSVC without the GUI (colmap#4955) - 78cc729 Fix default prior_position_loss_scale to sqrt(chi2_95_3dof) (colmap#4936) - 8653c6b Estimate two-view relative rotation covariance (colmap#4950) - e1317dd FlexGlue: match keypoints in the resized descriptor input frame (colmap#4949) - 3db6eb0 Extract ConvertPosePriorsToENU and skip pose priors without position (colmap#4937) - 77cbe48 Store and estimate rotation covariances on PoseGraph (colmap#4951) - 30a1a81 Support covariance whitening in CeresRotationAverager (colmap#4952) - e6644e7 Fix -Wunused-result warning in ThreadPool::AddTask (colmap#4958) - 5b6da88 Clear descriptor-index results for empty queries (colmap#4960) - d1d92d9 Add COVARIANCE reweighting for Ceres rotation averaging (colmap#4953) - 761d90d Enable covariance-weighted rotation averaging in global pipelines (colmap#4823) - fdee2e1 Reuse the pose graph of GlobalPipeline in the component mappers (colmap#4912) AI disclosure: Totally vibe coded
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
FlexGlue resizes each input image to a fixed descriptor resolution (
512×512by default), and during training its mutual nearest-neighbor (MNN) search andmax_distance_pxthreshold operate in that resized frame.Previously, the exported matcher denormalized predicted correspondences back to each image's original
(W, H)pixel coordinates and scaledmax_distance_pxby(max(W0, H0) / 512 + max(W1, H1) / 512) / 2(see cvg/FlexGlue#16):max_distance_pxwere distorted across axes relative to the512×512feature grid (e.g., 50% looser along the short axis of a 3:2 image, and inconsistent on portrait–landscape pairs).This PR rescales keypoints linearly from
(W, H)to(descriptor_width, descriptor_height)inKeypointsToTensorDataand passesdescriptor_sizeasimage_size0/image_size1to the ONNX matcher:kpts / image_size * 2 - 1) is unchanged.max_distance_pxnow operate directly in the512×512descriptor input frame (pixel_scale = max(512, 512) / 512 = 1) using the already-releasedv0.1ONNX weights, and give identical results with models exported both before and after Fix distance threshold frame cvg/FlexGlue#16.Evaluation
End-to-end evaluation with DaD keypoints (2048 and 4096) and default FlexGlue options (
min_covisibility = 0.5,early_stop_decay = 0.5,filter_threshold = 0.1), averaged over 3 reconstruction seeds (mean ± std), with matching speed measured on an idle A100 GPU:ETH3D DSLR (25 scenes, 898 images, 22,569 pairs)
mainmain160.9 → 161.7at 2k,309.0 → 312.3at 4k) while raw matches drop, raising the inlier ratio (78.5% → 81.2%at 2k,75.7% → 78.3%at 4k) and reducing geometric verification time by 21–26%.IMC 2023 (7 scenes, 261 images, 10,935 pairs)
mainmainheritage/dioscuri:+2.96[email protected]° at 2k,+1.10[email protected]° /+1.97AUC@5° at 4k), and geometric verification is 19–23% faster.heritage/wall(34 images,6048×4032), where registering all 34/34 views across all 3 seeds (vs. 2/3 seeds onmain) raiseswallAUC@10°(+1.74) while 4 marginal wide-baseline views are registered with coarse poses that lowerwall[email protected]°, alongside tightening the short-axis MNN window on 3:2 images from4.5to3.0resized pixels (heritage/cyprus24 → 23registered at 4k).Tests
FlexGlueTest.MatchTranslatedImagesto verify that anisotropically rescaling an image's camera dimensions and keypoints (4W × 2H) leaves the matches unchanged.