Repository navigation
Commit 24b4c54
[fix](inverted index) Compute each visible term frequency of a segment once
### What problem does this PR solve?
Issue Number: None
Related PR: #69698
Problem Summary:
On a merge-on-write segment with hidden rows, the document frequency of a term
without its hidden rows costs an intersection of the term's postings with the
hidden rows. The terms a scored phrase prefix expands to were asked for twice:
once to choose the terms to keep, and once to add up their frequencies, so with
the default of 50 common expansions and dense hidden rows their postings were
traversed twice per segment, and a read of the dictionary with a larger limit
or a second prefix repeated the work again.
cached_visible_doc_freq computes the frequency once per term for the segment
and the field, and both steps use it, in CLucene and in SNII.
### Release note
None
### Check List (For Author)
- Test: Unit Test, Regression test, Manual test
- phrase_prefix_query_test: the document frequency of each term is read
once across choosing and counting.
- test_match_phrase_prefix_score passes unchanged on V2, V3 and SNII.
- Manual: the merge-on-write table with a hidden `apple ra` scores the
same rows at 1.0498 before and after a full compaction.
- Behavior changed: No
- Does this need documentation: No
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>1 parent 59f3dc7 commit 24b4c54
3 files changed
Lines changed: 67 additions & 18 deletions
File tree
- be
- src/storage/index/inverted/reader
- test/storage/index/inverted/query/engine
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
254 | 254 | | |
255 | 255 | | |
256 | 256 | | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
257 | 275 | | |
258 | 276 | | |
259 | 277 | | |
| |||
575 | 593 | | |
576 | 594 | | |
577 | 595 | | |
578 | | - | |
579 | | - | |
580 | | - | |
581 | | - | |
582 | | - | |
583 | | - | |
584 | | - | |
585 | | - | |
586 | | - | |
| 596 | + | |
| 597 | + | |
587 | 598 | | |
588 | 599 | | |
589 | 600 | | |
| |||
719 | 730 | | |
720 | 731 | | |
721 | 732 | | |
722 | | - | |
723 | | - | |
724 | | - | |
725 | | - | |
726 | | - | |
727 | | - | |
728 | | - | |
729 | | - | |
730 | | - | |
| 733 | + | |
| 734 | + | |
731 | 735 | | |
732 | 736 | | |
733 | 737 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
300 | 300 | | |
301 | 301 | | |
302 | 302 | | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
303 | 311 | | |
304 | 312 | | |
305 | 313 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
569 | 569 | | |
570 | 570 | | |
571 | 571 | | |
| 572 | + | |
| 573 | + | |
| 574 | + | |
| 575 | + | |
| 576 | + | |
| 577 | + | |
| 578 | + | |
| 579 | + | |
| 580 | + | |
572 | 581 | | |
573 | 582 | | |
574 | 583 | | |
| |||
614 | 623 | | |
615 | 624 | | |
616 | 625 | | |
| 626 | + | |
| 627 | + | |
| 628 | + | |
| 629 | + | |
| 630 | + | |
| 631 | + | |
| 632 | + | |
| 633 | + | |
| 634 | + | |
| 635 | + | |
| 636 | + | |
| 637 | + | |
| 638 | + | |
| 639 | + | |
| 640 | + | |
| 641 | + | |
| 642 | + | |
| 643 | + | |
| 644 | + | |
| 645 | + | |
| 646 | + | |
| 647 | + | |
| 648 | + | |
| 649 | + | |
| 650 | + | |
| 651 | + | |
| 652 | + | |
| 653 | + | |
617 | 654 | | |
618 | 655 | | |
619 | 656 | | |
| |||
0 commit comments