Skip to content

flann: use per-thread storage for heap pool to remove lock contention#29346

Merged
asmorkalov merged 1 commit into
opencv:4.xfrom
Ijtihed:fix/flann-heap-pool-tls
Jun 26, 2026
Merged

flann: use per-thread storage for heap pool to remove lock contention#29346
asmorkalov merged 1 commit into
opencv:4.xfrom
Ijtihed:fix/flann-heap-pool-tls

Conversation

@Ijtihed

@Ijtihed Ijtihed commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Fixes #25281

cvflann::Heap<T>::getPooledInstance() keeps its reusable per-search heap in a single process-wide static map guarded by a static cv::Mutex. the pool here is keyed by cv::utils::getThreadID() at every call site (kdtree, kmeans and hierarchical index search) so each thread only ever touches its own entry but the issue is every search on every thread still has to take that one global lock. Under a multi-threaded FlannBasedMatcher workload the searches serialize on it which is why FlannBasedMatcher ended up slower than BFMatcher (in the issue it measured CPU utilization dropping from ~2000% to ~300%).

This moves the pool into per-thread storage (cv::TLSData) and drops the mutex so that each thread own an independent set of heaps and there is no shared state to lock. I didnt change the per-thread heap semantics so the use_count guard and the iteration-threshold cleanup now simply operate on the calling thread's own map. cv::TLSData is used instead ofa raw thread_local because a thread_local for this dynamically-initialized object seemed unreliable to me in testing and also its the primitive cv::utils::getThreadID() itself is built on aswell.

a shared FlannBasedMatcher hit by radiusMatch from N threads on an Intel Core i5-12450H (8c/12t) goes from ~2200 to ~3700 matches/s at 8 threads (~1.7x) and single-thread cost stays unchanged. The old code stopped scaling after ~4 threads while the new one keeps scaling and gets higher with higher core count :)

I added tests to cover heap ordering, capacity, the per-thread isolation guarantee and concurrent use (threaded cases guarded by OPENCV_DISABLE_THREAD_SUPPORT)

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
  • The PR is proposed to the proper branch
  • There is a reference to the original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake

@Ijtihed

Ijtihed commented Jun 21, 2026

Copy link
Copy Markdown
Contributor Author

I think the failures are repo wide @asmorkalov lmk what I should do, thanks

@asmorkalov asmorkalov added this to the 4.14.0 milestone Jun 22, 2026
@asmorkalov

Copy link
Copy Markdown
Contributor

cvflann::Heap<T>::getPooledInstance() is designed for global key-value store, not for thead local stuff and may be used in different ways. The modification changes behaviour. I propose not to modify the Heap class itself, but replace
getPooledInstance() call with C++ thead_local variable inside the function. It changes local behaviour, but not change API behaviour and much simpler.

Places where the function is used:

$ grep -Rin getPooledInstance
flann/include/opencv2/flann/kdtree_index.h:452:        const cv::Ptr<Heap<BranchSt>>& heap = Heap<BranchSt>::getPooledInstance(cv::utils::getThreadID(), (int)size_);
flann/include/opencv2/flann/kmeans_index.h:531:            const cv::Ptr<Heap<BranchSt>>& heap = Heap<BranchSt>::getPooledInstance(cv::utils::getThreadID(), (int)size_);
flann/include/opencv2/flann/heap.h:188:    static cv::Ptr<Heap<T>> getPooledInstance(
flann/include/opencv2/flann/hierarchical_clustering_index.h:535:        const cv::Ptr<Heap<BranchSt>>& heap = Heap<BranchSt>::getPooledInstance(cv::utils::getThreadID(), (int)size_);

@Ijtihed
Ijtihed force-pushed the fix/flann-heap-pool-tls branch from 1bc2fa9 to fbd4dc6 Compare June 25, 2026 18:19
@Ijtihed

Ijtihed commented Jun 25, 2026

Copy link
Copy Markdown
Contributor Author

cvflann::Heap<T>::getPooledInstance() is designed for global key-value store, not for thead local stuff and may be used in different ways. The modification changes behaviour. I propose not to modify the Heap class itself, but replace getPooledInstance() call with C++ thead_local variable inside the function. It changes local behaviour, but not change API behaviour and much simpler.

Places where the function is used:

$ grep -Rin getPooledInstance
flann/include/opencv2/flann/kdtree_index.h:452:        const cv::Ptr<Heap<BranchSt>>& heap = Heap<BranchSt>::getPooledInstance(cv::utils::getThreadID(), (int)size_);
flann/include/opencv2/flann/kmeans_index.h:531:            const cv::Ptr<Heap<BranchSt>>& heap = Heap<BranchSt>::getPooledInstance(cv::utils::getThreadID(), (int)size_);
flann/include/opencv2/flann/heap.h:188:    static cv::Ptr<Heap<T>> getPooledInstance(
flann/include/opencv2/flann/hierarchical_clustering_index.h:535:        const cv::Ptr<Heap<BranchSt>>& heap = Heap<BranchSt>::getPooledInstance(cv::utils::getThreadID(), (int)size_);

Followed :) please take another look when you're free

@asmorkalov
asmorkalov self-requested a review June 25, 2026 18:40
@asmorkalov asmorkalov self-assigned this Jun 26, 2026
@asmorkalov
asmorkalov merged commit da4a417 into opencv:4.x Jun 26, 2026
28 of 31 checks passed
@asmorkalov asmorkalov mentioned this pull request Jul 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

static cv::Mutex in function getPooledInstance() hurt multithreading perfermance

2 participants