Skip to content

[8.2] Fix HNSW vector index ghost memory leak (MOD-13761)#8850

Merged
meiravgri merged 2 commits into8.2from
meiravg_x_inc_edges-8.2
Mar 26, 2026
Merged

[8.2] Fix HNSW vector index ghost memory leak (MOD-13761)#8850
meiravgri merged 2 commits into8.2from
meiravg_x_inc_edges-8.2

Conversation

@meiravgri
Copy link
Copy Markdown
Collaborator

@meiravgri meiravgri commented Mar 26, 2026 •
edited by cursor Bot
Loading

backport #8843 to 8.2 (backport vecsim to v8.2.12)

Mark if applicable

  • This PR introduces API changes
  • This PR introduces serialization changes

Release Notes

  • This PR requires release notes
  • This PR does not require release notes

If a release note is required (bug fix / new feature / enhancement), describe the user impact of this PR in the title.

If a release note is required (bug fix / new feature / enhancement), describe the user impact of this PR in the title.

Note

Low Risk
The provided diff contains no code changes, so it should be low risk; verify the actual backport commit includes the intended HNSW/VecSim memory fix before merging.

Overview
No functional changes are visible in the provided diff (+++ /dev/null only), so there is no user-impacting fix to review or release-note to ship from this PR as-is.

If this is meant to backport the HNSW “ghost memory leak” fix, the actual VecSim/HNSW changes are missing from the diff and should be reattached before merge.

Written by Cursor Bugbot for commit 4a40e8f. This will update automatically on new commits. Configure here.

@meiravgri meiravgri requested a review from alonre24 March 26, 2026 05:20
@jit-ci
Copy link
Copy Markdown

jit-ci Bot commented Mar 26, 2026 •
edited
Loading

🛡️ Jit Security Scan Results

CRITICAL HIGH MEDIUM

✅ No security findings were detected in this PR

Security scan by Jit

@meiravgri meiravgri enabled auto-merge March 26, 2026 05:41
@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud Bot commented Mar 26, 2026

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@meiravgri meiravgri added this pull request to the merge queue Mar 26, 2026
@fcostaoliveira
Copy link
Copy Markdown
Contributor

fcostaoliveira commented Mar 26, 2026

Automated performance analysis summary

This comment was automatically generated given there is performance data available.

In summary:

  • Detected a total of 5 stable tests between versions.

You can check a comparison in detail via the grafana link

##

Details

Tests with No Significant Changes (5 tests)

Tests with No Significant Changes

Test Case Baseline 8.2 (median obs. +- std.dev) Comparison meiravg_x_inc_edges-8.2 (median obs. +- std.dev) % change (higher-better) Note
ftsb-10K-enwiki_abstract-hashes-term-suffix 2752 +- 0.7% (3 datapoints) 2802.0 1.8% No Change
search-ftsb-1700K-docs-union-iterators-q3 7.7 +- 2.3% (7 datapoints) 7.9 3.3% potential IMPROVEMENT
search-high-cardinality-negation-term-comparison_union_all_other_terms 8.9 +- 5.7% (7 datapoints) 8.9 -0.9% No Change
vecsim-arxiv-titles-384-angular-filters-m16-ef-128-fulltext-filter 624 +- 4.7% (7 datapoints) 658.0 5.4% potential IMPROVEMENT
vecsim-arxiv-titles-384-angular-filters-m16-ef-128-tag-filter 82406 +- 1.4% (7 datapoints) 81747.0 -0.8% No Change

@fcostaoliveira
Copy link
Copy Markdown
Contributor

fcostaoliveira commented Mar 26, 2026

Automated performance analysis summary

This comment was automatically generated given there is performance data available.

In summary:

  • Detected a total of 7 stable tests between versions.

You can check a comparison in detail via the grafana link

##

Details

Tests with No Significant Changes (7 tests)

Tests with No Significant Changes

Test Case Baseline 8.2 (median obs. +- std.dev) Comparison meiravg_x_inc_edges-8.2 (median obs. +- std.dev) % change (higher-better) Note
ftsb-10K-enwiki_abstract-hashes-term-suffix 2752 +- 0.7% (3 datapoints) 2802.0 1.8% No Change
search-ftsb-1700K-docs-union-iterators-q3 7.7 +- 2.3% (7 datapoints) 7.9 3.3% potential IMPROVEMENT
search-ftsb-370K-docs-union-iterators-q4 8.3 +- 1.1% (3 datapoints) 8.2 -1.0% No Change
search-high-cardinality-negation-term-comparison_union_all_other_terms 8.9 +- 5.7% (7 datapoints) 8.9 -0.9% No Change
vecsim-arxiv-titles-384-angular-filters-m16-ef-128-fulltext-filter 624 +- 4.7% (7 datapoints) 658.0 5.4% potential IMPROVEMENT
vecsim-arxiv-titles-384-angular-filters-m16-ef-128-numeric-filter 165 +- 5.6% (7 datapoints) 153.0 -7.4% potential REGRESSION
vecsim-arxiv-titles-384-angular-filters-m16-ef-128-tag-filter 82406 +- 1.4% (7 datapoints) 81747.0 -0.8% No Change

@fcostaoliveira
Copy link
Copy Markdown
Contributor

fcostaoliveira commented Mar 26, 2026 •
edited
Loading

Automated performance analysis summary

This comment was automatically generated given there is performance data available.

In summary:

  • Detected a total of 30 stable tests between versions.
  • Detected a total of 8 highly unstable benchmarks (8 baseline).
  • Latency analysis confirmed regressions in 3 of the unstable tests:
  • Detected a total of 2 improvements above the improvement water line.
  • Detected a total of 1 regressions bellow the regression water line 8.0%.

You can check a comparison in detail via the grafana link

Performance Improvements - Comparison between 8.2 and meiravg_x_inc_edges-8.2.

Time Period from a month ago. (environment used: oss-standalone)

Test Case Baseline 8.2 (median obs. +- std.dev) Comparison meiravg_x_inc_edges-8.2 (median obs. +- std.dev) % change (higher-better) Note
ftsb-10K-enwiki_pages-hashes-fulltext-mixed_simple-1word-query_write_1_to_read_20.yml 1081 +- 8.2% (7 datapoints) 1183 9.4% waterline=8.2%. IMPROVEMENT
ftsb-10K-enwiki_abstract-hashes-term-suffix-withsuffixtrie 85562 +- 6.0% (3 datapoints) 92883 8.6% IMPROVEMENT

Performance Regressions and Issues - Comparison between 8.2 and meiravg_x_inc_edges-8.2.

Time Period from a month ago. (environment used: oss-standalone)

Test Case Baseline 8.2 (median obs. +- std.dev) Comparison meiravg_x_inc_edges-8.2 (median obs. +- std.dev) % change (higher-better) Note
search-numeric-sortby 4643 +- 32.5% UNSTABLE (3 datapoints) 2171 -53.2% UNSTABLE (baseline high variance); server: FT.SEARCH p50 increased 132.5% (baseline CV=38.9%); client: Latency increased 145.6% (baseline CV=39.5%)
ftsb-1M-enwiki_abstract-hashes-fulltext-2word-intersection-query-non-sortable 65 +- 34.2% UNSTABLE (3 datapoints) 43 -34.5% UNSTABLE (baseline high variance); server: FT.SEARCH p50 decreased 11.7% (baseline CV=18.0%); client: OverallQuantiles.allCommands.q50 decreased 11.6% (baseline CV=11.1%); neither server nor client side confirms regression
ftsb-1M-enwiki_abstract-hashes-fulltext-2word-intersection-query 386 +- 1.8% (3 datapoints) 282 -26.9% REGRESSION
search-filtering-tag-numeric 237 +- 10.8% UNSTABLE (7 datapoints) 214 -9.8% UNSTABLE (baseline high variance); server: FT.AGGREGATE p50 increased 10.9% (baseline CV=10.7%); client: Latency increased 12.0% (baseline CV=6.5%)
search-numeric 2654 +- 34.8% UNSTABLE (3 datapoints) 2590 -2.4% UNSTABLE (baseline high variance); server: FT.SEARCH p50 increased 68.8% (baseline CV=42.4%); client: Latency increased 66.1% (baseline CV=41.1%)
ftsb-1M-enwiki_abstract-hashes-fulltext-2word-union-query 3613 +- 14.4% UNSTABLE (7 datapoints) 3624 0.3% UNSTABLE (baseline high variance); server: p50 latency stable; client: client latency stable; neither server nor client side confirms regression
ftsb-1M-enwiki_abstract-hashes-fulltext-simple-1word-query 944 +- 14.9% UNSTABLE (7 datapoints) 980 3.9% UNSTABLE (baseline high variance); server: p50 latency stable; client: client latency stable; neither server nor client side confirms regression
ftsb-10K-enwiki_abstract-hashes-term-wildcard 15931 +- 11.2% UNSTABLE (7 datapoints) 17314 8.7% UNSTABLE (baseline high variance); server: FT.SEARCH p50 decreased 7.5% (baseline CV=10.6%); client: client latency stable; neither server nor client side confirms regression
search-numeric-sortby-desc 2214 +- 42.8% UNSTABLE (7 datapoints) 4911 121.8% UNSTABLE (baseline high variance); server: FT.SEARCH p50 decreased 56.7% (baseline CV=22.8%); client: Latency decreased 53.8% (baseline CV=18.9%); neither server nor client side confirms regression
Tests with No Significant Changes (30 tests)

Tests with No Significant Changes

Test Case Baseline 8.2 (median obs. +- std.dev) Comparison meiravg_x_inc_edges-8.2 (median obs. +- std.dev) % change (higher-better) Note
ftsb-10K-enwiki_abstract-hashes-fulltext-sortby 93 +- 1.8% (7 datapoints) 90.00 -2.9% No Change
ftsb-10K-enwiki_abstract-hashes-term-prefix 9124 +- 4.3% (7 datapoints) 8413.00 -7.8% potential REGRESSION
ftsb-10K-enwiki_abstract-hashes-term-suffix 2752 +- 0.7% (3 datapoints) 2802.00 1.8% No Change
ftsb-10K-enwiki_pages-hashes-load 66166 +- 6.6% (3 datapoints) 64463.00 -2.6% No Change
ftsb-10K-multivalue-numeric-json 991 +- 1.0% (7 datapoints) 995.00 0.4% No Change
ftsb-10K-singlevalue-numeric-json 452 +- 0.9% (7 datapoints) 459.00 1.5% No Change
ftsb-1K-enwiki_abstract-hashes-term-contains 2333 +- 1.3% (7 datapoints) 2251.00 -3.5% potential REGRESSION
ftsb-1M-enwiki_abstract-hashes-fulltext-2word-union-query-non-sortable 1220 +- 7.2% (7 datapoints) 1314.00 7.7% potential IMPROVEMENT
ftsb-1M-enwiki_abstract-hashes-load 24351 +- 3.5% (7 datapoints) 23666.00 -2.8% No Change
ftsb-1M-nyc_taxis-ftadd-load 30794 +- 2.1% (7 datapoints) 31319.00 1.7% No Change
ftsb-1M-nyc_taxis-hashes-load 34580 +- 0.9% (3 datapoints) 35018.00 1.3% No Change
search-aggregate-post-filter-simple.yml 128629 +- 3.9% (7 datapoints) 129717.00 0.8% No Change
search-filtering-tag-numeric-filter-pipeline 25362 +- 2.4% (7 datapoints) 25376.00 0.1% No Change
search-ftsb-10K-enwiki_abstract-hashes-term-withoutsuffix-trie 51786 +- 3.7% (3 datapoints) 52457.00 1.3% No Change
search-ftsb-10K-enwiki_abstract-hashes-term-withsuffix-trie 49740 +- 2.7% (3 datapoints) 52800.00 6.2% potential IMPROVEMENT
search-ftsb-1700K-docs-union-iterators-q3 7.7 +- 2.3% (7 datapoints) 7.90 3.3% potential IMPROVEMENT
search-ftsb-1M-enwiki_abstract-hashes-fulltext-simple-1word-query-non-sortable 173 +- 2.4% (3 datapoints) 184.00 6.3% potential IMPROVEMENT
search-ftsb-1M-enwiki_abstract-hashes-gc 506 +- 5.6% (7 datapoints) 506.00 0.0% No Change
search-ftsb-370K-docs-union-iterators-q4 8.3 +- 1.1% (3 datapoints) 8.20 -1.0% No Change
search-ftsb-5200K-docs-union-iterators-q1 0.83 +- 2.4% (7 datapoints) 0.83 0.0%
search-ftsb-5500K-docs-union-iterators-q2 1.2 +- 1.5% (7 datapoints) 1.20 4.3% potential IMPROVEMENT
search-geo 218 +- 3.4% (7 datapoints) 215.00 -1.4% No Change
search-high-cardinality-negation-term-baseline 29 +- 4.0% (7 datapoints) 27.00 -6.7% potential REGRESSION
search-high-cardinality-negation-term-comparison_union_all_other_terms 8.9 +- 5.7% (7 datapoints) 8.90 -0.9% No Change
search-numeric-optimize 13479 +- 1.1% (3 datapoints) 13716.00 1.8% No Change
search-numeric-sortby-desc-optimize 25 +- 5.1% (7 datapoints) 26.00 3.6% potential IMPROVEMENT
search-numeric-sortby-optimize 24 +- 5.5% (7 datapoints) 24.00 0.3% No Change
vecsim-arxiv-titles-384-angular-filters-m16-ef-128-fulltext-filter 624 +- 4.7% (7 datapoints) 658.00 5.4% potential IMPROVEMENT
vecsim-arxiv-titles-384-angular-filters-m16-ef-128-numeric-filter 165 +- 5.6% (7 datapoints) 153.00 -7.4% potential REGRESSION
vecsim-arxiv-titles-384-angular-filters-m16-ef-128-tag-filter 82406 +- 1.4% (7 datapoints) 81747.00 -0.8% No Change

Merged via the queue into 8.2 with commit c639d25 Mar 26, 2026
35 checks passed
@meiravgri meiravgri deleted the meiravg_x_inc_edges-8.2 branch March 26, 2026 14:12
@YaacovHazan YaacovHazan mentioned this pull request May 5, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@alonre24 alonre24 alonre24 approved these changes

Assignees

No one assigned

Labels

action:run-benchmark size:XS

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@meiravgri @fcostaoliveira @alonre24