Skip to content

MOD-7976: Fix HFE Regression#5318

Merged
kei-nan merged 6 commits intomasterfrom
master_jk_fix_hfe_regression
Dec 10, 2024
Merged

MOD-7976: Fix HFE Regression#5318
kei-nan merged 6 commits intomasterfrom
master_jk_fix_hfe_regression

Conversation

@kei-nan
Copy link
Collaborator

@kei-nan kei-nan commented Dec 5, 2024

Describe the changes in the pull request

Use new expiration API to avoid regression issues with HFE support when expiration isn't being used

  1. Before: Using RM_Call was expensive and as a result our indexing took a hit even when expiration isn't used
  2. Change: Using a new redis API to avoid using RM_Call and efficently check if a hash key has any fields with expiration before collecting the expiration times.
  3. Outcome: Faster indexing time, fixing the regression that was introduced with HFE PR.

Main objects this PR modified

  1. HFE

Mark if applicable

  • This PR introduces API changes
  • This PR introduces serialization changes

@kei-nan kei-nan requested review from nafraf and removed request for nafraf December 5, 2024 14:42
@kei-nan kei-nan self-assigned this Dec 8, 2024
@kei-nan kei-nan requested review from oshadmi and raz-mon and removed request for raz-mon December 8, 2024 09:40
@codecov
Copy link

codecov bot commented Dec 8, 2024

Codecov Report

Attention: Patch coverage is 33.33333% with 6 lines in your changes missing coverage. Please review.

Project coverage is 86.62%. Comparing base (708dc5b) to head (bf62df6).
Report is 3 commits behind head on master.

Files with missing lines Patch % Lines
src/document_basic.c 28.57% 5 Missing ⚠️
src/debug_commands.c 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5318      +/-   ##
==========================================
- Coverage   86.72%   86.62%   -0.11%     
==========================================
  Files         193      193              
  Lines       34704    34681      -23     
==========================================
- Hits        30097    30041      -56     
- Misses       4607     4640      +33     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Collaborator

@raz-mon raz-mon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great!

oshadmi
oshadmi previously approved these changes Dec 9, 2024
@kei-nan kei-nan force-pushed the master_jk_fix_hfe_regression branch from 4ff16d5 to bf62df6 Compare December 10, 2024 12:37
Copy link
Collaborator

@oshadmi oshadmi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice 💪🏼
(ignoring patch codecov for now until we have an appropriate Redis branch)

@kei-nan kei-nan added this pull request to the merge queue Dec 10, 2024
Merged via the queue into master with commit bebdeb7 Dec 10, 2024
@kei-nan kei-nan deleted the master_jk_fix_hfe_regression branch December 10, 2024 14:49
@redisearch-backport-pull-request
Copy link
Contributor

Backport failed for 2.10, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin 2.10
git worktree add -d .worktree/backport-5318-to-2.10 origin/2.10
cd .worktree/backport-5318-to-2.10
git switch --create backport-5318-to-2.10
git cherry-pick -x bebdeb70a2437266a09b5bdf1f86f3b3ae701a2b

@redisearch-backport-pull-request
Copy link
Contributor

Backport failed for 8.0, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin 8.0
git worktree add -d .worktree/backport-5318-to-8.0 origin/8.0
cd .worktree/backport-5318-to-8.0
git switch --create backport-5318-to-8.0
git cherry-pick -x bebdeb70a2437266a09b5bdf1f86f3b3ae701a2b

kei-nan added a commit that referenced this pull request Dec 10, 2024
* * use new expiration API to avoid regression issues with HFE support when expiration isn't being used

* * update sdk to get new hash field expiration API

* * set monitorFieldExpiration default based on the existence of `RedisModule_HashFieldMinExpire`

* * skip field expiration tests if below 8.0 API

* * Code Review Comments
* Prevent turning on the monitorFieldExpiration if the API does not exist

* * move to run against redis 8.0

(cherry picked from commit bebdeb7)
kei-nan added a commit that referenced this pull request Dec 10, 2024
MOD-7976: Fix HFE Regression (#5318)

* * use new expiration API to avoid regression issues with HFE support when expiration isn't being used

* * update sdk to get new hash field expiration API

* * set monitorFieldExpiration default based on the existence of `RedisModule_HashFieldMinExpire`

* * skip field expiration tests if below 8.0 API

* * Code Review Comments
* Prevent turning on the monitorFieldExpiration if the API does not exist

* * move to run against redis 8.0

(cherry picked from commit bebdeb7)
kei-nan added a commit that referenced this pull request Dec 10, 2024
MOD-7976: Fix HFE Regression (#5318)

* * use new expiration API to avoid regression issues with HFE support when expiration isn't being used

* * update sdk to get new hash field expiration API

* * set monitorFieldExpiration default based on the existence of `RedisModule_HashFieldMinExpire`

* * skip field expiration tests if below 8.0 API

* * Code Review Comments
* Prevent turning on the monitorFieldExpiration if the API does not exist

* * move to run against redis 8.0

(cherry picked from commit bebdeb7)
kei-nan added a commit that referenced this pull request Dec 10, 2024
MOD-7976: Fix HFE Regression (#5318)

* * use new expiration API to avoid regression issues with HFE support when expiration isn't being used

* * update sdk to get new hash field expiration API

* * set monitorFieldExpiration default based on the existence of `RedisModule_HashFieldMinExpire`

* * skip field expiration tests if below 8.0 API

* * Code Review Comments
* Prevent turning on the monitorFieldExpiration if the API does not exist

* * move to run against redis 8.0

(cherry picked from commit bebdeb7)
kei-nan added a commit that referenced this pull request Dec 11, 2024
MOD-7976: Fix HFE Regression (#5318)

* * use new expiration API to avoid regression issues with HFE support when expiration isn't being used

* * update sdk to get new hash field expiration API

* * set monitorFieldExpiration default based on the existence of `RedisModule_HashFieldMinExpire`

* * skip field expiration tests if below 8.0 API

* * Code Review Comments
* Prevent turning on the monitorFieldExpiration if the API does not exist

* * move to run against redis 8.0

(cherry picked from commit bebdeb7)
@nafraf nafraf mentioned this pull request Dec 17, 2024
2 tasks
github-merge-queue bot pushed a commit that referenced this pull request Dec 18, 2024
…returning them as valid results (#5118)

* MOD-7329: Determine if fields and documents are expired before returning them as valid results (#4858)

* * initial commit

* * fix missing case in test_expire.py

* * fix coord compilation

* * fix doc info test

* * try and fix some of the tests

* * release iterator

* * do not set expiration time in the context of the coordinator since it doesn't have an index spec

* * support c++ tests that do not provide spec for iterators

* * no point in passing timestamp in coordinator flow

* * redis 7.4 version in ci is actually 7.3, for codecov we need our tests to run

* * reduce max memory from + 200000 to 150000

* * fix test

* * cleanup code

* * Code Review: Round #1

* * Add geoshape tests, only pass index instead of pointer to filter ctx

* * Add vector test

* * Code Review - Round #2

* * try and fix tests

* * Code Review - Round #3

* * Fix Sanitizer Build

* * Code Review - Round #4

* * Minor change in debug command SET_MONITOR_EXPIRATION

* * Code Review - Round #5

* * fix tests

* * Code Review - Round #6

* * fix crash

* * fix test

* * improve field mask translation time

* fix compilation

* fix compilation error

* handle field mask of 64 bit

* * Code Review - Round #7

* * fix opening of key edge cases

* * free fieldIdToIndex array

* Code Review - Round #9

* * fix rebase conflicts
* support wild card reader filtering

* * mac os: fallback to CLOCK_REALTIME if CLOCK_REALTIME_COARSE is not defined

(cherry picked from commit 4a67a87)

* MOD-7979: Protect Against RS_FIELDMASK_ALL Mask Edge Case (#5128)

* * check mask edge case

* * Code Review - Round #1

(cherry picked from commit 2c037bc)

* [8.0] MOD-7976: Fix HFE Regression (#5336)

MOD-7976: Fix HFE Regression (#5318)

* * use new expiration API to avoid regression issues with HFE support when expiration isn't being used

* * update sdk to get new hash field expiration API

* * set monitorFieldExpiration default based on the existence of `RedisModule_HashFieldMinExpire`

* * skip field expiration tests if below 8.0 API

* * Code Review Comments
* Prevent turning on the monitorFieldExpiration if the API does not exist

* * move to run against redis 8.0

(cherry picked from commit bebdeb7)

* * rebase fixes

* * add REDISMODULE_OPEN_KEY_NOEFFECTS to OpenKey when loading json document

* * fix for test_debug_commands.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants