Skip to content

[8.2] MOD-14342: Fix property order discrepancy in FILTER#8662

Merged
raz-mon merged 3 commits into8.2from
backport-MOD-14065-to-8.2
Mar 30, 2026
Merged

[8.2] MOD-14342: Fix property order discrepancy in FILTER#8662
raz-mon merged 3 commits into8.2from
backport-MOD-14065-to-8.2

Conversation

@raz-mon
Copy link
Copy Markdown
Collaborator

@raz-mon raz-mon commented Mar 11, 2026

Describe the changes in the pull request

Backport of #8548 to 8.2 - fixes property order discrepancy in FILTER.

  1. Current: During indexing, if a FILTER expression references properties in a certain order, documents may be incorrectly excluded when properties appear in a different order than in the filter expression.
  2. Change: Introduces EVAL_MODE_INDEX and EVAL_MODE_QUERY to distinguish between indexing and query-time evaluation. In EVAL_MODE_INDEX, missing properties no longer cause errors but return undefined values.
  3. Outcome: Consistent behavior regardless of property order in filter expressions during indexing.

Which additional issues this PR fixes

  1. MOD-14342
  2. Includes fix from Wait for index to avoid flakiness in new filter tests #8627: Added waitForIndex calls to prevent test flakiness

Main objects this PR modified

  1. src/aggregate/expr/expression.c - Core evaluation logic with eval mode distinction
  2. tests/pytests/test_filter.py - Added regression tests
  3. tests/cpptests/test_cpp_expr.cpp - Added C++ unit tests

Mark if applicable

  • This PR introduces API changes
  • This PR introduces serialization changes

Release Notes

  • This PR requires release notes
  • This PR does not require release notes

8.2-specific adaptations

  • Used QUERY_ENOPROPKEY / QUERY_ENOPROPVAL instead of QUERY_ERROR_CODE_* (8.2 naming)
  • Used {0} initialization for QueryError (no QueryError_Default() in 8.2)
  • Used eval->err->code == QUERY_OK instead of QueryError_IsOk() (not available in 8.2)
  • Used e->lookupObj->name directly instead of RLookupKey_GetName() (not available in 8.2)
  • Kept legacy RLookup_LoadRuleFields signature (no status parameter in 8.2)
  • Removed src/pipeline/pipeline_construction.c changes (file doesn't exist in 8.2)

Note

Medium Risk
Changes core expression evaluation for index-time FILTER predicates, altering how missing fields and type mismatches are handled; could affect which documents are indexed for some filters. Behavior is covered by new C++ and Python regression tests but touches central evaluation logic.

Overview
Fixes an indexing bug where FILTER expressions could include/exclude documents depending on the order of referenced fields when some fields were missing.

Adds EvalMode to expression evaluation and makes index-time evaluation (EVAL_MODE_INDEX) treat missing properties as a non-match (false) instead of an error, while keeping query/FT.ADD IF evaluation strict (EVAL_MODE_QUERY). Predicate evaluation was refactored to centralize missing-field handling and preserve correct short-circuiting for &&/||.

Adds regression tests (C++ and Python) for AND/OR with missing fields, missing-vs-NULL comparisons, and numeric type-mismatch behavior in filters.

Written by Cursor Bugbot for commit 827c25d. This will update automatically on new commits. Configure here.

@jit-ci
Copy link
Copy Markdown

jit-ci Bot commented Mar 11, 2026

🛡️ Jit Security Scan Results

CRITICAL HIGH MEDIUM

✅ No security findings were detected in this PR


Security scan by Jit

Comment thread src/aggregate/expr/expression.c Outdated
Comment thread src/aggregate/expr/expression.c Outdated
Comment thread src/aggregate/expr/expression.c Outdated
@raz-mon raz-mon changed the title MOD-14342: Backport MOD-14065 - Fix property order discrepancy in FILTER to 8.2 [8.2] MOD-14342: Backport MOD-14065 - Fix property order discrepancy in FILTER Mar 11, 2026
@raz-mon raz-mon changed the title [8.2] MOD-14342: Backport MOD-14065 - Fix property order discrepancy in FILTER [8.2] MOD-14342: Fix property order discrepancy in FILTER Mar 11, 2026
@raz-mon raz-mon force-pushed the backport-MOD-14065-to-8.2 branch from 21dff77 to 0237a3a Compare March 12, 2026 10:59
Comment thread src/aggregate/expr/expression.c
Comment thread src/rules.c
* WIP make the order of the properties in the FILTER expression not matter for evaluation

* New approach

* Handle missing fields comparison

* Enhance readability

* Enhance tests

* Fix initialization and cleanup

* fix tests

* Remove unused APIs

* Fix leak in test

* Address review

* Explicitly set mode of query evaluators

* Fix leak in other test

* Address review

* small proposal to PR (#8587)

* Revert accidental code removal

---------

Co-authored-by: Joan Fontanals <[email protected]>
@raz-mon raz-mon force-pushed the backport-MOD-14065-to-8.2 branch from 0237a3a to 62fd32f Compare March 12, 2026 15:00
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Comment thread src/aggregate/expr/expression.c
- Revert EvalCtx_Eval return codes back to REDISMODULE_ERR (not EXPR_EVAL_ERR)
- Remove testEvalCtxEvalExprNullExpr and testEvalCtxEvalExprUnknownProperty tests
- Revert accidental submodule updates to match 8.2 branch
@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 15, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.95%. Comparing base (0ab6437) to head (827c25d).
⚠️ Report is 11 commits behind head on 8.2.

Additional details and impacted files
@@            Coverage Diff             @@
##              8.2    #8662      +/-   ##
==========================================
- Coverage   88.96%   88.95%   -0.02%     
==========================================
  Files         260      260              
  Lines       41990    41994       +4     
  Branches     3851     3851              
==========================================
- Hits        37357    37356       -1     
- Misses       4584     4589       +5     
  Partials       49       49              
Flag Coverage Δ
flow 82.02% <97.67%> (-0.48%) ⬇️
unit 46.81% <72.09%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Add QueryError_ClearError(&r->status) before EvalCtx_Destroy(r) to prevent
leaking error messages that may be set during filter evaluation (e.g., type
mismatches in RSValue_Cmp).
@sonarqubecloud
Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
34.0% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

@raz-mon
Copy link
Copy Markdown
Collaborator Author

raz-mon commented Mar 15, 2026

Approved by me.

@raz-mon raz-mon added this pull request to the merge queue Mar 30, 2026
Merged via the queue into 8.2 with commit 3078aa3 Mar 30, 2026
33 of 34 checks passed
@raz-mon raz-mon deleted the backport-MOD-14065-to-8.2 branch March 30, 2026 08:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants