Skip to content

MOD-11190: Add IndexSpec RDB Comparison Tests - Fix Index RDB field initialization#6817

Merged
kei-nan merged 12 commits intomasterfrom
master_jk_add_rdb_index_spec_tests
Sep 21, 2025
Merged

MOD-11190: Add IndexSpec RDB Comparison Tests - Fix Index RDB field initialization#6817
kei-nan merged 12 commits intomasterfrom
master_jk_add_rdb_index_spec_tests

Conversation

@kei-nan
Copy link
Collaborator

@kei-nan kei-nan commented Sep 11, 2025

We noticed some fields do not get initialized correctly when the index spec is initialized from an RDB.
Added mock rdb support for tests and added tests to compare an index to itself after it stored to an RDB and loaded again.

A clear and concise description of what the PR is solving, including:

  1. Current: Missing index spec struct members initialization
  2. Change: Create one function to initialize all regular members and call it from both the new function and when loading from an RDB.
  3. Outcome: More correct index initialization when loading an index from an RDB.

Main objects this PR modified

  1. Index Spec

Mark if applicable

  • This PR introduces API changes
  • This PR introduces serialization changes

cursor[bot]

This comment was marked as outdated.

@codecov
Copy link

codecov bot commented Sep 11, 2025

Codecov Report

❌ Patch coverage is 96.82540% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.72%. Comparing base (a4c9723) to head (cc02dbc).
⚠️ Report is 4 commits behind head on master.

Files with missing lines Patch % Lines
src/spec.c 96.82% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6817      +/-   ##
==========================================
- Coverage   86.76%   86.72%   -0.04%     
==========================================
  Files         288      288              
  Lines       46302    46309       +7     
  Branches     9195     9195              
==========================================
- Hits        40173    40162      -11     
- Misses       5978     5996      +18     
  Partials      151      151              
Flag Coverage Δ
flow 84.67% <96.82%> (-0.16%) ⬇️
unit 49.92% <60.31%> (+0.37%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@kei-nan kei-nan requested a review from JoanFM September 14, 2025 06:01
JoanFM
JoanFM previously approved these changes Sep 14, 2025
cursor[bot]

This comment was marked as outdated.

cursor[bot]

This comment was marked as outdated.

@kei-nan kei-nan enabled auto-merge September 15, 2025 15:43
cursor[bot]

This comment was marked as outdated.

src/spec.c Outdated
int16_t numFields) {
sp->flags = flags;
sp->numFields = numFields;
sp->fields = rm_calloc(sizeof(FieldSpec), numFields);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we also add the index error initialization loop here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When creating an index it would mean doing an init 1024 times.
Seems a bit excessive, no?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not if we attempt to free all of the 1024 later... how are we avoiding crashes there at the moment? We change the sp->numFields later on to the actual number?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Allocating 1024 fields is excessive on its own; maybe we can avoid that completely?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll try passing 0 and see if the unit tests pass , don't want to cause an issue elsewhere in the code but I guess it would be better behaviour.
I think we do a realloc when creating the first field anyhow - so the duration in the code we would be without a field is small.
Wondering if a schema with zero fields is even possible.

@kei-nan kei-nan force-pushed the master_jk_add_rdb_index_spec_tests branch from e28ff31 to 0ce401c Compare September 17, 2025 10:36
@kei-nan kei-nan requested a review from GuyAv46 September 17, 2025 10:40
cursor[bot]

This comment was marked as outdated.

@kei-nan kei-nan added this pull request to the merge queue Sep 21, 2025
Merged via the queue into master with commit a73f3f7 Sep 21, 2025
18 checks passed
@kei-nan kei-nan deleted the master_jk_add_rdb_index_spec_tests branch September 21, 2025 12:25
@redisearch-backport-pull-request
Copy link
Contributor

Backport failed for 2.8, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin 2.8
git worktree add -d .worktree/backport-6817-to-2.8 origin/2.8
cd .worktree/backport-6817-to-2.8
git switch --create backport-6817-to-2.8
git cherry-pick -x a73f3f7c57c49c3d24ae86d6ed9b4fafd32ae419

@redisearch-backport-pull-request
Copy link
Contributor

Backport failed for 2.6, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin 2.6
git worktree add -d .worktree/backport-6817-to-2.6 origin/2.6
cd .worktree/backport-6817-to-2.6
git switch --create backport-6817-to-2.6
git cherry-pick -x a73f3f7c57c49c3d24ae86d6ed9b4fafd32ae419

@redisearch-backport-pull-request
Copy link
Contributor

Backport failed for 2.10, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin 2.10
git worktree add -d .worktree/backport-6817-to-2.10 origin/2.10
cd .worktree/backport-6817-to-2.10
git switch --create backport-6817-to-2.10
git cherry-pick -x a73f3f7c57c49c3d24ae86d6ed9b4fafd32ae419

redisearch-backport-pull-request bot pushed a commit that referenced this pull request Sep 21, 2025
…nitialization (#6817)

* initial commit

* prepare code for CI run

* move terms and fieldIdToIndex initialization to  initializeIndexSpec

* cleanup test a bit

* remove extra RWlock init (#6823)

* fix code review comments

* remove print from mock

* remove print from mock functions

* code review comment

* remove log remainder

* switch rm_calloc arg order

* add initializeFieldSpec function

---------

Co-authored-by: Joan Fontanals <[email protected]>
(cherry picked from commit a73f3f7)
@redisearch-backport-pull-request
Copy link
Contributor

github-merge-queue bot pushed a commit that referenced this pull request Sep 21, 2025
…ield initialization (#6866)

MOD-11190: Add IndexSpec RDB Comparison Tests - Fix Index RDB field initialization (#6817)

* initial commit

* prepare code for CI run

* move terms and fieldIdToIndex initialization to  initializeIndexSpec

* cleanup test a bit

* remove extra RWlock init (#6823)

* fix code review comments

* remove print from mock

* remove print from mock functions

* code review comment

* remove log remainder

* switch rm_calloc arg order

* add initializeFieldSpec function

---------


(cherry picked from commit a73f3f7)

Co-authored-by: kei-nan <[email protected]>
Co-authored-by: Joan Fontanals <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants