Repository navigation
Conversation
sarlinpe
force-pushed
the
sarlinpe/flexglue-inference-semaphore
branch
from
November 5, 2026 13:09
1b25dcc to
5009431
Compare
ahojnnes
approved these changes
Nov 8, 2026
ahojnnes
left a comment
Contributor
There was a problem hiding this comment.
Reviewed the diff at 5009431 (stacked on 4957 at ebd27cb) and verified it merges cleanly onto the fixed 4957 head (28a748a, no conflicts; the only overlapping file is flexglue_test.cc). The limiter's slot assignment, permit lifetime, replica-session creation, and CUDA provider options check out, and the (N, M) benchmark shows bit-identical match counts. No new significant findings; please rebase onto 4957 before merging. Ship.
sarlinpe
force-pushed
the
sarlinpe/flexglue-inference-semaphore
branch
from
November 8, 2026 20:33
5009431 to
8e3958f
Compare
sarlinpe
enabled stacked auto-merge (squash)
November 8, 2026 20:34
fringehush-gif
added a commit
to fringehush-gif/colmap
that referenced
this pull request
Nov 9, 2026
## Summary Bring in upstream's shared FlexGlue workers and bounded GPU inference slots. CPU preprocessing can overlap while inference concurrency stays independently configurable within each worker group. Preserve the fork's deferred matching-result replacement and CUDA output regression. ## Changes - Merge the ONNX inference limiter, shared model ownership, worker/inference options and bindings. - Keep both upstream limiter tests and the existing dynamic-shape CUDA regression when resolving onnx_test.cc. - Give the setup-failure regression an explicit missing matcher path when model downloads are disabled, so it reaches the intended descriptor-loading error. - Regressions cover limiter capacity/release, option bounds, worker setup errors and preservation of existing matching results on failure. ## Upstream commits - df19970 - FlexGlue: share one matcher across multiple worker threads per GPU (colmap#4957) - 3c1a1b9 - FlexGlue: bound concurrent GPU inferences with a semaphore (colmap#4969) ## Validation - 42 focused native tests passed without skips: 14 ONNX, 2 FlexGlue option and 26 matching/controller cases. CUDA EP initialization occurred inside the GPU setup-error case; the dynamic-shape CUDA output regression passed. - clang-tidy-18 completed for six affected production/test translation units with no displayed source diagnostics; generated/header warnings were suppressed. Diff and added-line formatting checks passed. - Incremental qualification reused 47 production objects and 13 private archives; seven remaining translation units compiled and the CLI linked. Link/provider checks and five help/version smokes passed. The exported-source version reports Unknown commit/date. - Real FlexGlue model inference, Python and Windows are not qualified by these checks. AI disclosure: Totally vibe coded
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #4957.
Summary
Decouples CPU image decoding/preprocessing parallelism ($N$ ) from concurrent GPU model execution ($M$ ) in
FlexGlueMatchingOptions::num_workers_per_gpu,FlexGlueMatchingOptions::num_inferences_per_gpu,FlexGlue:ONNXInferenceLimiter(util/onnx.{h,cc}): Slot-assigning counting semaphore bounding concurrent GPU inferences across a larger pool of worker threads, paired withOrtCUDAProviderOptionsV2(arena_extend_strategy=kSameAsRequested,use_ep_level_unified_stream=1) and lazy per-slot CUDAOrt::Sessionreplicas (allow_session_replicas=falseonFlexGlue's 344 MB dense descriptor so its weights are never duplicated).FlexGlueMatchingOptions::num_inferences_per_gpu(feature/flexglue.{h,cc}, CLI,pycolmap, docs): Bounds concurrentdescriptor_+matcher_executions per CUDA GPU (defaultnum_workers_per_gpu = 4,num_inferences_per_gpu = 2), and releases decoded full-resolutionBitmapbuffers immediately afterResizeBilinearToCHWso workers waiting on the semaphore hold only the resized512x512input tensor (~3 MBinstead of72–150 MB).Benchmark:$(N, M)$ Grid Search on
delivery_area(44 images @ 24 MP, 946 pairs, NVIDIA A100-SXM4-40GB)FLEXGLUE(K=2048keypoints)workers)inferences)1142.941.00x46.7456.1%9.5%186 / 304 W1,964 MiB169,5813132.991.30x36.8570.0%12.2%226 / 310 W1,960 MiB169,5816130.841.39x34.6374.6%13.0%239 / 311 W1,960 MiB169,5812229.611.45x33.3663.4%17.3%249 / 398 W2,788 MiB169,5814225.461.69x29.2471.6%19.7%286 / 407 W2,788 MiB169,5813326.861.60x30.5766.4%22.2%272 / 421 W3,600 MiB169,5816324.401.76x28.2271.4%24.2%289 / 417 W3,610 MiB169,5818323.671.81x27.4073.6%24.7%296 / 419 W3,616 MiB169,5814426.341.63x30.1367.0%25.0%276 / 415 W4,412 MiB169,5818424.171.78x27.9772.8%26.3%291 / 414 W4,440 MiB169,5816627.091.59x30.8966.9%25.8%272 / 416 W6,038 MiB169,5818827.891.54x31.7166.6%24.5%266 / 422 W7,708 MiB169,581121229.231.47x33.0465.6%20.6%253 / 417 W10,982 MiB169,581