Skip to content

[MOD-13606] Enable LTO in all CI workflows#8092

Draft
gdesmott wants to merge 3 commits intomasterfrom
gd_lto_ci
Draft

[MOD-13606] Enable LTO in all CI workflows#8092
gdesmott wants to merge 3 commits intomasterfrom
gd_lto_ci

Conversation

@gdesmott
Copy link
Collaborator

@gdesmott gdesmott commented Jan 20, 2026 •
edited
Loading

Enable LTO in the CI:

  • Build on Platforms (only Ubuntu Noble atm)
  • PR
  • Build Artifacts
  • Benchmarks

Mark if applicable

  • This PR introduces API changes
  • This PR introduces serialization changes

@gdesmott
ci: updated to LLVM 21 on Linux
fe11cb8 
That's the version currently required to build with LTO.

Also: move llvm version to its own file so we can have the version in a single place.
@gdesmott gdesmott mentioned this pull request Jan 20, 2026
Merged
2 tasks
@jit-ci
Copy link

jit-ci bot commented Jan 20, 2026

Hi, I’m Jit, a friendly security platform designed to help developers build secure applications from day zero with an MVS (Minimal viable security) mindset.

In case there are security findings, they will be communicated to you as a comment inside the PR.

Hope you’ll enjoy using Jit.

Questions? Comments? Want to learn more? Get in touch with us.

@fcostaoliveira
Copy link
Contributor

fcostaoliveira commented Jan 20, 2026 •
edited
Loading

Automated performance analysis summary

This comment was automatically generated given there is performance data available.

In summary:

  • Detected a total of 22 stable tests between versions.
  • Detected a total of 6 highly unstable benchmarks (6 baseline).
  • Latency analysis confirmed regressions in 1 of the unstable tests:
  • Detected a total of 13 improvements above the improvement water line.
  • Detected a total of 2 regressions bellow the regression water line 8.0%.

You can check a comparison in detail via the grafana link

Performance Improvements - Comparison between master and gd_lto_ci.

Time Period from 30 days ago. (environment used: oss-standalone)

Test Case Baseline master (median obs. +- std.dev) Comparison gd_lto_ci (median obs. +- std.dev) % change (higher-better) Note
search-numeric 2432 +- 3.8% (2 datapoints) 3801.0 56.3% IMPROVEMENT
ftsb-10K-enwiki_abstract-hashes-fulltext-sortby 72 +- 7.1% (7 datapoints) 90.0 25.2% IMPROVEMENT
search-numeric-sortby-optimize 25 +- 6.1% (7 datapoints) 31.0 22.0% IMPROVEMENT
search-ftsb-1M-enwiki_abstract-hashes-fulltext-simple-1word-query-non-sortable 164 +- 6.9% (2 datapoints) 196.0 20.0% IMPROVEMENT
search-ftsb-10K-enwiki_abstract-hashes-fulltext-aggregate-sortby-limit-0-100 831 +- 1.7% (7 datapoints) 992.0 19.4% IMPROVEMENT
ftsb-10K-enwiki_abstract-hashes-term-prefix 5905 +- 4.1% (7 datapoints) 6763.0 14.5% IMPROVEMENT
ftsb-1M-nyc_taxis-ftadd-load 28487 +- 6.1% (7 datapoints) 32552.0 14.3% IMPROVEMENT
ftsb-10K-enwiki_abstract-hashes-term-wildcard 8415 +- 7.0% (7 datapoints) 9528.0 13.2% IMPROVEMENT
ftsb-1M-enwiki_abstract-hashes-fulltext-2word-union-query 2961 +- 7.4% (7 datapoints) 3341.0 12.8% IMPROVEMENT
ftsb-1M-enwiki_abstract-hashes-load 21504 +- 7.9% (7 datapoints) 24009.0 11.6% IMPROVEMENT
ftsb-10K-enwiki_pages-hashes-fulltext-mixed_simple-1word-query_write_1_to_read_20.yml 996 +- 7.4% (7 datapoints) 1106.0 11.1% IMPROVEMENT
search-ftsb-1700K-docs-union-iterators-q3 7.9 +- 0.6% (7 datapoints) 8.6 9.5% IMPROVEMENT
vecsim-arxiv-titles-384-angular-filters-m16-ef-128-fulltext-filter 582 +- 6.1% (7 datapoints) 636.0 9.2% IMPROVEMENT

Performance Regressions and Issues - Comparison between master and gd_lto_ci.

Time Period from 30 days ago. (environment used: oss-standalone)

Test Case Baseline master (median obs. +- std.dev) Comparison gd_lto_ci (median obs. +- std.dev) % change (higher-better) Note
search-numeric-sortby-desc 3696 +- 19.2% UNSTABLE (7 datapoints) 2062 -44.2% UNSTABLE (baseline high variance); server: FT.SEARCH p50 increased 82.7% (baseline CV=27.9%); client: Latency increased 79.3% (baseline CV=26.8%)
search-filtering-tag-numeric 277 +- 9.7% (7 datapoints) 239 -13.6% waterline=9.7%. REGRESSION
search-numeric-sortby-desc-optimize 31 +- 7.6% (7 datapoints) 28 -8.4% REGRESSION
ftsb-1M-enwiki_abstract-hashes-fulltext-2word-union-query-non-sortable 981 +- 12.9% UNSTABLE (7 datapoints) 1002 2.1% UNSTABLE (baseline high variance); server: p50 latency stable; client: client latency stable; neither server nor client side confirms regression
vecsim-arxiv-titles-384-angular-filters-m16-ef-128-numeric-filter 153 +- 11.7% UNSTABLE (7 datapoints) 158 2.9% UNSTABLE (baseline high variance); server: p50 latency stable; client: client latency stable; neither server nor client side confirms regression
ftsb-1M-enwiki_abstract-hashes-fulltext-2word-intersection-query 392 +- 16.0% UNSTABLE (7 datapoints) 450 14.7% UNSTABLE (baseline high variance); server: p50 latency stable; client: OverallQuantiles.allCommands.q50 decreased 10.8% (baseline CV=14.3%); neither server nor client side confirms regression
ftsb-1M-enwiki_abstract-hashes-fulltext-2word-intersection-query-non-sortable 39 +- 20.2% UNSTABLE (7 datapoints) 46 17.5% UNSTABLE (baseline high variance); server: FT.SEARCH p50 decreased 7.4% (baseline CV=13.2%); client: OverallQuantiles.allCommands.q50 increased 6.5% (baseline CV=8.7%); neither server nor client side confirms regression
ftsb-1M-enwiki_abstract-hashes-fulltext-simple-1word-query 735 +- 23.8% UNSTABLE (7 datapoints) 1154 57.0% UNSTABLE (baseline high variance); server: FT.SEARCH p50 decreased 36.3% (baseline CV=16.2%); client: OverallQuantiles.allCommands.q50 decreased 33.5% (baseline CV=16.1%); neither server nor client side confirms regression
Tests with No Significant Changes (22 tests)

Tests with No Significant Changes

Test Case Baseline master (median obs. +- std.dev) Comparison gd_lto_ci (median obs. +- std.dev) % change (higher-better) Note
ftsb-10K-enwiki_abstract-hashes-term-suffix 2254 +- 0.5% (7 datapoints) 2301.00 2.1% No Change
ftsb-10K-enwiki_abstract-hashes-term-suffix-withsuffixtrie 16328 +- 1.4% (7 datapoints) 17223.00 5.5% potential IMPROVEMENT
ftsb-10K-enwiki_pages-hashes-load 59301 +- 6.6% (7 datapoints) 63548.00 7.2% potential IMPROVEMENT
ftsb-10K-multivalue-numeric-json 988 +- 1.2% (7 datapoints) 1056.00 6.9% potential IMPROVEMENT
ftsb-10K-singlevalue-numeric-json 485 +- 0.1% (2 datapoints) 513.00 5.8% potential IMPROVEMENT
ftsb-1K-enwiki_abstract-hashes-term-contains 1913 +- 1.5% (7 datapoints) 1951.00 2.0% No Change
ftsb-1M-nyc_taxis-hashes-load 30591 +- 5.2% (7 datapoints) 32148.00 5.1% potential IMPROVEMENT
search-aggregate-post-filter-simple.yml 17518 +- 1.2% (7 datapoints) 17660.00 0.8% No Change
search-filtering-tag-numeric-filter-pipeline 10902 +- 0.9% (7 datapoints) 11606.00 6.5% potential IMPROVEMENT
search-ftsb-10K-enwiki_abstract-hashes-fulltext-search-sortby-limit-0-100 924 +- 3.6% (2 datapoints) 920.00 -0.4% No Change
search-ftsb-10K-enwiki_abstract-hashes-term-withoutsuffix-trie 14568 +- 0.6% (7 datapoints) 14678.00 0.8% No Change
search-ftsb-10K-enwiki_abstract-hashes-term-withsuffix-trie 14402 +- 2.4% (2 datapoints) 14985.00 4.0% potential IMPROVEMENT
search-ftsb-1M-enwiki_abstract-hashes-fulltext-simple-1word-query-one-indexed-field 7371 +- 2.9% (2 datapoints) 7587.00 2.9% No Change
search-ftsb-370K-docs-union-iterators-q4 8.2 +- 0.8% (7 datapoints) 8.60 4.6% potential IMPROVEMENT
search-ftsb-5200K-docs-union-iterators-q1 0.83 +- 0.6% (7 datapoints) 0.88 6.0% potential IMPROVEMENT
search-ftsb-5500K-docs-union-iterators-q2 1.2 +- 1.1% (7 datapoints) 1.20 4.2% potential IMPROVEMENT
search-geo 223 +- 0.3% (2 datapoints) 240.00 7.7% potential IMPROVEMENT
search-high-cardinality-negation-term-baseline 38 +- 2.1% (7 datapoints) 38.00 0.6% No Change
search-high-cardinality-negation-term-comparison_union_all_other_terms 14 +- 4.0% (7 datapoints) 15.00 7.9% potential IMPROVEMENT
search-numeric-optimize 8302 +- 1.1% (7 datapoints) 8501.00 2.4% No Change
search-numeric-sortby 2382 +- 2.3% (2 datapoints) 2306.00 -3.2% potential REGRESSION
vecsim-arxiv-titles-384-angular-filters-m16-ef-128-tag-filter 15774 +- 1.4% (7 datapoints) 16296.00 3.3% potential IMPROVEMENT

@gdesmott
ci: benchmark: enable LTO
d0a4bea
@gdesmott
ci: enable LTO on Ubuntu Noble
08897bf 
That's the only platform with a new enough toolchain for now.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

action:run-benchmark ci:all-platforms size:S

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@gdesmott @fcostaoliveira