Skip to content

[MOD-7949] Avoid lazy expire upon scan keys in background#5274

Merged
alonre24 merged 9 commits intomasterfrom
avoid_lazy_expire_upon_scan_keys
Dec 2, 2024
Merged

[MOD-7949] Avoid lazy expire upon scan keys in background#5274
alonre24 merged 9 commits intomasterfrom
avoid_lazy_expire_upon_scan_keys

Conversation

@alonre24
Copy link
Collaborator

@alonre24 alonre24 commented Nov 29, 2024

Describe the changes in the pull request

Avoid lazy expired during background scan of keys upon creating the index to avoid potential propagation of multiple DEL commands within a transaction (that might cause cross slot violation down the road).

We disabled the monitoring of has field expiration by default since otherwise, we call HPEXPIRETIME on every indexing
of a document, which causes lazy expiration even if not intended (this should be changed anyway since it was causing performance regressions)

Mark if applicable

  • This PR introduces API changes
  • This PR introduces serialization changes

@alonre24 alonre24 marked this pull request as ready for review December 1, 2024 11:07
@codecov
Copy link

codecov bot commented Dec 1, 2024

Codecov Report

Attention: Patch coverage is 75.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 86.68%. Comparing base (98cfcbf) to head (a8a0c91).
Report is 4 commits behind head on master.

Files with missing lines Patch % Lines
src/spec.c 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5274      +/-   ##
==========================================
- Coverage   86.70%   86.68%   -0.03%     
==========================================
  Files         193      193              
  Lines       34752    34752              
==========================================
- Hits        30133    30126       -7     
- Misses       4619     4626       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

DvirDukhan
DvirDukhan previously approved these changes Dec 1, 2024
@alonre24 alonre24 requested a review from kei-nan December 1, 2024 13:26
…nce ft.debug cannot be dispatched to shard via cluster connection
@alonre24 alonre24 added this pull request to the merge queue Dec 1, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Dec 1, 2024
@alonre24 alonre24 added this pull request to the merge queue Dec 2, 2024
Merged via the queue into master with commit 15ed082 Dec 2, 2024
@alonre24 alonre24 deleted the avoid_lazy_expire_upon_scan_keys branch December 2, 2024 10:48
@redisearch-backport-pull-request
Copy link
Contributor

Backport failed for 2.8, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin 2.8
git worktree add -d .worktree/backport-5274-to-2.8 origin/2.8
cd .worktree/backport-5274-to-2.8
git switch --create backport-5274-to-2.8
git cherry-pick -x 15ed0825e235323aafe7053be558dcd91ad2542d

@redisearch-backport-pull-request
Copy link
Contributor

Backport failed for 2.10, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin 2.10
git worktree add -d .worktree/backport-5274-to-2.10 origin/2.10
cd .worktree/backport-5274-to-2.10
git switch --create backport-5274-to-2.10
git cherry-pick -x 15ed0825e235323aafe7053be558dcd91ad2542d

@redisearch-backport-pull-request
Copy link
Contributor

Backport failed for 8.0, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin 8.0
git worktree add -d .worktree/backport-5274-to-8.0 origin/8.0
cd .worktree/backport-5274-to-8.0
git switch --create backport-5274-to-8.0
git cherry-pick -x 15ed0825e235323aafe7053be558dcd91ad2542d

env.cmd('DEBUG', 'SET-ACTIVE-EXPIRE', '0')
env.expect('HSET', 'doc:1', 't', 'bar').equal(1)
env.expect('HSET', 'doc:2', 't', 'arr').equal(1)
env.expect('EXPIRE', 'doc:1', '1').equal(1)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use hexpire to make the test faster.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants