Skip to content

[2.10] [MOD-10037] add default scorer in configuration (#7065)#7084

Merged
JoanFM merged 14 commits into2.10from
backport-7065-to-2.10
Oct 28, 2025
Merged

[2.10] [MOD-10037] add default scorer in configuration (#7065)#7084
JoanFM merged 14 commits into2.10from
backport-7065-to-2.10

Conversation

@JoanFM
Copy link
Collaborator

@JoanFM JoanFM commented Oct 17, 2025

  • feat: add default scorer in configuration

  • fix and test config setters and getters

  • test: add flow tests checking scorer behavior

  • fix: fix as per cursor comments

  • fix: fix as per code review

  • fix: change the init config code

  • test: fix the tests

  • test: refactor tests

  • fix memory leak in getter

  • test: add explicit test BM25STD is the default since start

  • validate default scorer at module init

  • fix validation against extensions

  • apply changes as per review

  • Update tests/cpptests/test_cpp_forkgc.cpp

  • refactor small setter

  • try to fix sanitizer leak

  • test: Known leaks when onModuleLoad fails

  • Update src/module-init/module-init.c


(cherry picked from commit b61c075)

Describe the changes in the pull request

A clear and concise description of what the PR is solving, including:

  1. Current: The current state briefly
  2. Change: What is the change
  3. Outcome: Adding the outcome

Which additional issues this PR fixes

  1. MOD-...
  2. #...

Main objects this PR modified

  1. ...

Mark if applicable

  • This PR introduces API changes
  • This PR introduces serialization changes

Note

Introduce configurable DEFAULT_SCORER (default TFIDF), validate and apply it when no scorer is specified, expose it in INFO, and add tests and init/mocks plumbing.

  • Configuration:
    • Add DEFAULT_SCORER config (setter/getter), gated by ENABLE_UNSTABLE_FEATURES, validated against registered scorers.
    • Include defaultScorer in RSConfig and in RSConfig_GetInfoString/FT.CONFIG GET.
  • Initialization/Extensions:
    • Initialize RSGlobalConfig.defaultScorer to DEFAULT_SCORER_NAME at module load; validate configured scorer after extensions load.
    • Expose Extensions_InitDone() for safe validation timing.
  • Query/Aggregate Execution:
    • Use RSGlobalConfig.defaultScorer when no SCORER provided in search/aggregate (aggregate_request.c, LLAPI).
    • Adjust optimizer to treat unspecified or default scorer as term scorer; add BM25STD handling.
  • INFO:
    • Report default_scorer in INFO runtime_configurations.
  • Tests/Mocks:
    • Add Python tests covering config behavior, defaults, extensions, and startup validation.
    • Update C++ tests/mocks to set/reset defaultScorer; add RMCK shutdown frees.
    • Minor test harness tweaks (bootstrap args, shutdown).

Written by Cursor Bugbot for commit 102fe93. This will update automatically on new commits. Configure here.

JoanFM and others added 2 commits October 17, 2025 09:31
* feat: add default scorer in configuration

* fix and test config setters and getters

* test: add flow tests checking scorer behavior

* fix: fix as per cursor comments

* fix: fix as per code review

* fix: change the init config code

* test: fix the tests

* test: refactor tests

* fix memory leak in getter

* test: add explicit test BM25STD is the default since start

* validate default scorer at module init

* fix validation against extensions

* apply changes as per review

* Update tests/cpptests/test_cpp_forkgc.cpp

* refactor small setter

* try to fix sanitizer leak

* test: Known leaks when onModuleLoad fails

* Update src/module-init/module-init.c

Co-authored-by: kei-nan <[email protected]>

---------

Co-authored-by: kei-nan <[email protected]>
(cherry picked from commit b61c075)
@JoanFM
Copy link
Collaborator Author

JoanFM commented Oct 17, 2025

Adding backport label because it should be easier to backport from already 2.10 to earlier versions than from master

@JoanFM JoanFM requested review from Itzikvaknin and kei-nan and removed request for kei-nan October 17, 2025 07:36
cursor[bot]

This comment was marked as outdated.

@JoanFM JoanFM force-pushed the backport-7065-to-2.10 branch 2 times, most recently from d0920e9 to 551acc9 Compare October 17, 2025 08:12
@JoanFM JoanFM force-pushed the backport-7065-to-2.10 branch from 551acc9 to e36f93e Compare October 17, 2025 08:21
@codecov
Copy link

codecov bot commented Oct 17, 2025

Codecov Report

❌ Patch coverage is 90.47619% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.25%. Comparing base (d8ac62a) to head (102fe93).
⚠️ Report is 5 commits behind head on 2.10.

Files with missing lines Patch % Lines
src/query_optimizer.c 33.33% 2 Missing ⚠️
src/aggregate/aggregate_request.c 66.66% 1 Missing ⚠️
src/config.c 95.45% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             2.10    #7084      +/-   ##
==========================================
- Coverage   89.31%   89.25%   -0.07%     
==========================================
  Files         207      207              
  Lines       36169    35304     -865     
==========================================
- Hits        32303    31509     -794     
+ Misses       3866     3795      -71     
Flag Coverage Δ
flow 83.82% <83.33%> (-0.14%) ⬇️
unit 42.47% <19.04%> (-0.25%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@JoanFM JoanFM requested a review from oshadmi October 22, 2025 10:59
cursor[bot]

This comment was marked as outdated.

@JoanFM JoanFM requested a review from nafraf October 27, 2025 17:24
cursor[bot]

This comment was marked as outdated.

@JoanFM JoanFM requested a review from nafraf October 28, 2025 08:25
@JoanFM JoanFM added this pull request to the merge queue Oct 28, 2025
Merged via the queue into 2.10 with commit 70e5a10 Oct 28, 2025
13 checks passed
@JoanFM JoanFM deleted the backport-7065-to-2.10 branch October 28, 2025 12:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants