Skip to content

[8.4] [MOD-12070] Extend indexing metrics#7760

Merged
redisearch-backport-pull-request[bot] merged 1 commit into8.4from
backport-7669-to-8.4
Dec 12, 2025
Merged

[8.4] [MOD-12070] Extend indexing metrics#7760
redisearch-backport-pull-request[bot] merged 1 commit into8.4from
backport-7669-to-8.4

Conversation

@redisearch-backport-pull-request
Copy link
Contributor

@redisearch-backport-pull-request redisearch-backport-pull-request bot commented Dec 11, 2025

Description

Backport of #7669 to 8.4.


Note

Adds per-field-type indexing operation counters and total documents-in-indexes metric, increments them during indexing, and exposes all via INFO MODULES with tests.

  • Metrics/Stats:
    • Add global counters in FieldsGlobalStats for per-type indexing ops: textTotalDocsIndexed, tagTotalDocsIndexed, numericTotalDocsIndexed, geoTotalDocsIndexed, geometryTotalDocsIndexed, vectorTotalDocsIndexed.
    • Implement FieldsGlobalStats_UpdateFieldDocsIndexed in info/global_stats.c and declare in info/global_stats.h.
    • Increment counters during indexing in src/document.c (fulltextPreprocessor and IndexerBulkAdd on success).
  • INFO MODULES output:
    • Expose per-type counters as total_indexing_ops_<type>_fields in AddToInfo_Fields.
    • Aggregate and expose total_num_docs_in_indexes via IndexesInfo_TotalInfo and AddToInfo_Indexes.
  • Indexes aggregation:
    • Track total_num_docs_in_indexes in TotalIndexesInfo by summing sp->stats.numDocuments.
  • Tests:
    • Add tests for total_num_docs_in_indexes and per-type total_indexing_ops_* metrics, including multi-value JSON behavior.

Written by Cursor Bugbot for commit a831b2a. This will update automatically on new commits. Configure here.

* Total num indexed metric

* total docs indexed by field type

* test total indexed metic

* test field metric

* format tests

* remove empty line

* remove unecessary wait

* meirav comments

* more comments + change metric name

* expose and test geometry

* Unify metric

* test multi json

(cherry picked from commit b346977)
case INDEXFLD_T_GEOMETRY:
RSGlobalStats.fieldsStats.geometryTotalDocsIndexed += toAdd;
break;
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Switch statement fails for multi-type fields' bitmask

The switch (field_type) statement uses exact equality matching against fs->types, but FieldType values are bitmasks that can be combined (e.g., INDEXFLD_T_NUMERIC | INDEXFLD_T_TAG = 0x0A). When a field has FieldSpec_Dynamic set (multiple types), fs->types will be a combination of flags that won't match any single case in the switch, causing the statistics to never be updated for dynamic/multi-type fields. The existing FieldsGlobalStats_UpdateStats function correctly uses if (fs->types & INDEXFLD_T_...) with bitwise AND operations.

Fix in Cursor Fix in Web

@codecov
Copy link

codecov bot commented Dec 11, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.93%. Comparing base (5aff644) to head (a831b2a).
⚠️ Report is 1 commits behind head on 8.4.

Additional details and impacted files
@@            Coverage Diff             @@
##              8.4    #7760      +/-   ##
==========================================
- Coverage   85.98%   85.93%   -0.05%     
==========================================
  Files         331      331              
  Lines       52969    53008      +39     
  Branches    11998    11998              
==========================================
+ Hits        45544    45555      +11     
- Misses       7258     7286      +28     
  Partials      167      167              
Flag Coverage Δ
flow 84.52% <100.00%> (-0.44%) ⬇️
unit 52.62% <56.41%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@redisearch-backport-pull-request redisearch-backport-pull-request bot added this pull request to the merge queue Dec 12, 2025
Merged via the queue into 8.4 with commit 53dec0a Dec 12, 2025
29 checks passed
@redisearch-backport-pull-request redisearch-backport-pull-request bot deleted the backport-7669-to-8.4 branch December 12, 2025 15:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant