Skip to content

Avoid adding connection to marked deleted elements upon delete in-place in tiered index [MOD-7732]#543

Merged
alonre24 merged 5 commits intomainfrom
tiered_avoid_updates_for_marked_deleted
Sep 9, 2024
Merged

Avoid adding connection to marked deleted elements upon delete in-place in tiered index [MOD-7732]#543
alonre24 merged 5 commits intomainfrom
tiered_avoid_updates_for_marked_deleted

Conversation

@alonre24
Copy link
Collaborator

@alonre24 alonre24 commented Sep 7, 2024

Describe the changes in the pull request

Upon in-place deletion of vectors that occur after asynchronous deletion in the tiered index, we did not properly handle the repairs for the "marked deleted" vectors, as we allowed updating new connections to such elements. These elements will not be accounted for in the asynchronous repair for the marked deleted vector and may cause corruption in the graph state along the way.
This PR fixes this by checking the elements selected upon repairing a node due to in-place delete and NOT updating new connections to deleted vectors.

Mark if applicable

  • This PR introduces API changes
  • This PR introduces serialization changes

@codecov
Copy link

codecov bot commented Sep 7, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.16%. Comparing base (a4015a5) to head (148699d).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #543   +/-   ##
=======================================
  Coverage   97.15%   97.16%           
=======================================
  Files          94       94           
  Lines        4846     4862   +16     
=======================================
+ Hits         4708     4724   +16     
  Misses        138      138           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@alonre24 alonre24 requested a review from GuyAv46 September 8, 2024 11:43
GuyAv46
GuyAv46 previously approved these changes Sep 8, 2024
@alonre24 alonre24 requested a review from meiravgri September 8, 2024 14:31
Copy link
Collaborator

@meiravgri meiravgri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch
few comments

@alonre24 alonre24 enabled auto-merge September 9, 2024 12:54
@alonre24 alonre24 added this pull request to the merge queue Sep 9, 2024
Merged via the queue into main with commit 3325068 Sep 9, 2024
@alonre24 alonre24 deleted the tiered_avoid_updates_for_marked_deleted branch September 9, 2024 14:18
github-actions bot pushed a commit that referenced this pull request Sep 9, 2024
…ce in tiered index [MOD-7732] (#543)

* test buggy scenario - WIP

* Fix logic for inplace, add test

* make test stricter + format

* avoid collecting marked deleted elements in the candidates selection phase.

* Meirav's CR

(cherry picked from commit 3325068)
@github-actions
Copy link

github-actions bot commented Sep 9, 2024

github-actions bot pushed a commit that referenced this pull request Sep 9, 2024
…ce in tiered index [MOD-7732] (#543)

* test buggy scenario - WIP

* Fix logic for inplace, add test

* make test stricter + format

* avoid collecting marked deleted elements in the candidates selection phase.

* Meirav's CR

(cherry picked from commit 3325068)
@github-actions
Copy link

github-actions bot commented Sep 9, 2024

github-merge-queue bot pushed a commit that referenced this pull request Sep 9, 2024
…in-place in tiered index [MOD-7732] (#545)

Avoid adding connection to marked deleted elements upon delete in-place in tiered index [MOD-7732] (#543)

* test buggy scenario - WIP

* Fix logic for inplace, add test

* make test stricter + format

* avoid collecting marked deleted elements in the candidates selection phase.

* Meirav's CR

(cherry picked from commit 3325068)

Co-authored-by: alonre24 <[email protected]>
github-merge-queue bot pushed a commit that referenced this pull request Sep 12, 2024
…in-place in tiered index [MOD-7732] (#544)

* Avoid adding connection to marked deleted elements upon delete in-place in tiered index [MOD-7732] (#543)

* test buggy scenario - WIP

* Fix logic for inplace, add test

* make test stricter + format

* avoid collecting marked deleted elements in the candidates selection phase.

* Meirav's CR

(cherry picked from commit 3325068)

* fix test for 0.7 so it won't use non exiting debug

---------

Co-authored-by: alonre24 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants