Skip to content

[8.4] MOD-12234: Fix totalDocsLen updates#7261

Merged
redisearch-backport-pull-request[bot] merged 2 commits into8.4from
backport-7227-to-8.4
Nov 9, 2025
Merged

[8.4] MOD-12234: Fix totalDocsLen updates#7261
redisearch-backport-pull-request[bot] merged 2 commits into8.4from
backport-7227-to-8.4

Conversation

@redisearch-backport-pull-request
Copy link
Contributor

@redisearch-backport-pull-request redisearch-backport-pull-request bot commented Nov 9, 2025

Description

Backport of #7227 to 8.4.


Note

Fix document length stats by switching deletes/replaces to DocTable_Pop with proper numDocuments/totalDocsLen adjustments, removing oldMd usage, and updating tests accordingly.

  • Core/Indexing:
    • Replace DocTable_Delete with DocTable_Pop in indexer.c, redisearch_api.c, and spec.c, updating numDocuments and totalDocsLen and calling DMD_Return.
    • Remove RSAddDocumentCtx.oldMd and its cleanup; adjust replace path to release popped DMD.
    • Clarify refcount semantics comment in DocTable_Pop.
  • Tests:
    • Update C++ test to use DocTable_Pop.
    • Add BM25 average document length validation; adjust expected BM25/BM25STD values.
    • Update VecSim hybrid tests’ expected scores to reflect scoring/stat changes.

Written by Cursor Bugbot for commit 9e427e2. This will update automatically on new commits. Configure here.

* Fix BM25STD underflow wraparound

* Fix comment

* Fix documentation and tweak test

* Address review

* Remove old test

* Skip cluster on expiration test

* Fix leak in test

* Switch to rust ii (#6958)

* Switch to Rust inverted index

* Remove encoder and decoder getter usage

It is no longer possible to get the encoders or decoders from the flags.
Instead they are coupled to the inverted index or its reader.

* Update `testNumericInverted`

* Update testIndexFlags

* Don't check buffer growth in test

There is no reason for the test to check the internals of these structures.

* Fix size test on flags

* Fix size test for numeric index

* Fix size test for tag index

* Fix sizes on LLapi

* Update fork tests to const

* C GC updates

* Update fork tests

* Don't free deltas twice

* Fix sizes in Python tests

* Recreate index to have compression turned on

* Remove impossible test

This test requires a mutable access to the inverted index in a reader
and direct access to the repair block call. Both of these are not
possible so the test is removed.

* Remove C benchmarks

* Move header out of future

* Remove from iterator benches

* Fix symbols for GC benchmark

* Update iterator benchmark FFI

* Get num_entries on index directly

* Trim bench dependencies

* Fix filters in CPP tests

The text inverted indices expect to have a field mask filter, not the
`None` case.

* Update debug assertions

It is now possible to have an inverted index with no blocks in it.

* Partially fix totalDocLen metric

* Fix

* Make test not pass by accident

* Remove var

* Update totalDocLen on deletions as well

* Fix tests

* Fix assertions

* Clean up unused old dmd

* Add assertions to llapi

* Remove Delete API

* Skip unnecesary cluster tests

---------

Co-authored-by: Pieter <[email protected]>
(cherry picked from commit 25e65ef)
@codecov
Copy link

codecov bot commented Nov 9, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.63%. Comparing base (a895366) to head (9e427e2).
⚠️ Report is 1 commits behind head on 8.4.

Additional details and impacted files
@@            Coverage Diff             @@
##              8.4    #7261      +/-   ##
==========================================
- Coverage   85.65%   85.63%   -0.03%     
==========================================
  Files         326      326              
  Lines       50943    50942       -1     
  Branches    10921    10921              
==========================================
- Hits        43635    43623      -12     
- Misses       7141     7153      +12     
+ Partials      167      166       -1     
Flag Coverage Δ
flow 84.17% <64.70%> (-0.15%) ⬇️
unit 51.73% <64.70%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@redisearch-backport-pull-request redisearch-backport-pull-request bot added this pull request to the merge queue Nov 9, 2025
Merged via the queue into 8.4 with commit 5e5e295 Nov 9, 2025
19 checks passed
@redisearch-backport-pull-request redisearch-backport-pull-request bot deleted the backport-7227-to-8.4 branch November 9, 2025 15:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants