Skip to content

[8.4] MOD-14065: Fix property order discrepancy in FILTER#8661

Merged
raz-mon merged 3 commits into8.4from
backport-MOD-14065-to-8.4
Mar 30, 2026
Merged

[8.4] MOD-14065: Fix property order discrepancy in FILTER#8661
raz-mon merged 3 commits into8.4from
backport-MOD-14065-to-8.4

Conversation

@raz-mon
Copy link
Copy Markdown
Collaborator

@raz-mon raz-mon commented Mar 11, 2026

Describe the changes in the pull request

Backport of #8548 to 8.4 - fixes property order discrepancy in FILTER.

  1. Current: During indexing, if a FILTER expression references properties in a certain order, documents may be incorrectly excluded when properties appear in a different order than in the filter expression.
  2. Change: Introduces EVAL_MODE_INDEX and EVAL_MODE_QUERY to distinguish between indexing and query-time evaluation. In EVAL_MODE_INDEX, missing properties no longer cause errors but return undefined values.
  3. Outcome: Consistent behavior regardless of property order in filter expressions during indexing.

Which additional issues this PR fixes

  1. MOD-14337
  2. Includes fix from Wait for index to avoid flakiness in new filter tests #8627: Added waitForIndex calls to prevent test flakiness

Main objects this PR modified

  1. src/aggregate/expr/expression.c - Core evaluation logic with eval mode distinction
  2. tests/pytests/test_filter.py - Added regression tests
  3. tests/cpptests/test_cpp_expr.cpp - Added C++ unit tests

Mark if applicable

  • This PR introduces API changes
  • This PR introduces serialization changes

Release Notes

  • This PR requires release notes
  • This PR does not require release notes

8.4-specific adaptations

  • Used QUERY_ENOPROPKEY / QUERY_ENOPROPVAL instead of QUERY_ERROR_CODE_* (8.4 naming)
  • Used {0} initialization for QueryError (no QueryError_Default() in 8.4)
  • Used eval->err->code == QUERY_OK instead of QueryError_IsOk() (not available in 8.4)
  • Used e->lookupObj->name directly instead of RLookupKey_GetName() (not available in 8.4)

Note

Medium Risk
Touches core expression evaluation and changes how missing fields/type mismatches are handled during indexing, which can affect what documents get indexed. Scope is contained to FILTER/expression evaluation and is covered by new C++/Python regression tests.

Overview
Release note: Fixes a bug where FT.CREATE ... FILTER could incorrectly include/exclude documents during indexing depending on the order of properties in the filter expression when some fields were missing or had type mismatches.

This introduces an explicit EvalMode (EVAL_MODE_INDEX vs EVAL_MODE_QUERY) and updates predicate/property evaluation so index-time filters treat missing properties as a non-match (false) without raising errors, while query-time/FT.ADD IF evaluation remains strict. Adds regression coverage for AND/OR ordering, missing-field comparisons, and numeric type-mismatch behavior.

Written by Cursor Bugbot for commit 8b426b1. This will update automatically on new commits. Configure here.

@jit-ci
Copy link
Copy Markdown

jit-ci Bot commented Mar 11, 2026

🛡️ Jit Security Scan Results

CRITICAL HIGH MEDIUM

✅ No security findings were detected in this PR


Security scan by Jit

@raz-mon raz-mon force-pushed the backport-MOD-14065-to-8.4 branch from b50fa51 to 2acdf81 Compare March 12, 2026 10:59
Comment thread src/rules.c
* WIP make the order of the properties in the FILTER expression not matter for evaluation

* New approach

* Handle missing fields comparison

* Enhance readability

* Enhance tests

* Fix initialization and cleanup

* fix tests

* Remove unused APIs

* Fix leak in test

* Address review

* Explicitly set mode of query evaluators

* Fix leak in other test

* Address review

* small proposal to PR (#8587)

* Revert accidental code removal

---------

Co-authored-by: Joan Fontanals <[email protected]>
@raz-mon raz-mon force-pushed the backport-MOD-14065-to-8.4 branch from 2acdf81 to db79b05 Compare March 12, 2026 15:01
- Revert EvalCtx_Eval return codes back to REDISMODULE_ERR (not EXPR_EVAL_ERR)
- Remove testEvalCtxEvalExprNullExpr and testEvalCtxEvalExprUnknownProperty tests
- Revert accidental submodule updates to match 8.4 branch
@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 15, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.53%. Comparing base (8f0d8cb) to head (8b426b1).
⚠️ Report is 14 commits behind head on 8.4.

Additional details and impacted files
@@            Coverage Diff             @@
##              8.4    #8661      +/-   ##
==========================================
- Coverage   85.53%   85.53%   -0.01%     
==========================================
  Files         337      337              
  Lines       53354    53358       +4     
  Branches    11023    11023              
==========================================
+ Hits        45639    45640       +1     
- Misses       7572     7575       +3     
  Partials      143      143              
Flag Coverage Δ
flow 84.47% <97.56%> (-0.11%) ⬇️
unit 51.12% <68.29%> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Add QueryError_ClearError(&r->status) before EvalCtx_Destroy(r) to prevent
leaking error messages that may be set during filter evaluation (e.g., type
mismatches in RSValue_Cmp).
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Comment thread src/aggregate/expr/expression.c
@sonarqubecloud
Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
34.1% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

@raz-mon
Copy link
Copy Markdown
Collaborator Author

raz-mon commented Mar 15, 2026

Approved by me.

@raz-mon raz-mon added this pull request to the merge queue Mar 30, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Mar 30, 2026
@raz-mon raz-mon added this pull request to the merge queue Mar 30, 2026
Merged via the queue into 8.4 with commit 4986078 Mar 30, 2026
33 of 34 checks passed
@raz-mon raz-mon deleted the backport-MOD-14065-to-8.4 branch March 30, 2026 11:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants