Repository navigation
Delta Snapshot - Delete processing with new schema - #1784
AndreKurait merged 16 commits into
Conversation
…ter model for the different operations Signed-off-by: Andre Kurait <[email protected]>
Signed-off-by: Andre Kurait <[email protected]>
Signed-off-by: Andre Kurait <[email protected]>
Signed-off-by: Andre Kurait <[email protected]>
Signed-off-by: Andre Kurait <[email protected]>
Signed-off-by: Andre Kurait <[email protected]>
Signed-off-by: Andre Kurait <[email protected]>
Signed-off-by: Andre Kurait <[email protected]>
Signed-off-by: Andre Kurait <[email protected]>
peternied
left a comment
There was a problem hiding this comment.
Looks like a couple thinks are still in flight, seeing good progress here, a couple of comments on the current stand of the PR
| throw new IllegalArgumentException("Failed to parse source doc: " + e.getMessage(), e); | ||
| } | ||
| } | ||
| // TODO: We can get a performance improvement by keeping doc.source as a byte array until here |
There was a problem hiding this comment.
How did you want to handle these todos, in this PR or as follow up jiras?
There was a problem hiding this comment.
This is documenting existing behavior. Let's leave it here until we determine there's value in that refactor
There was a problem hiding this comment.
Please create jira ticket(s) so we can prioritize this work on the backlog or as OE tasks
Signed-off-by: Andre Kurait <[email protected]>
Signed-off-by: Andre Kurait <[email protected]>
Signed-off-by: Andre Kurait <[email protected]>
Signed-off-by: Andre Kurait <[email protected]>
…with new RFS schema Signed-off-by: Andre Kurait <[email protected]>
peternied
left a comment
There was a problem hiding this comment.
Minor comments, feel free to merge after they are addressed or deferred.
Signed-off-by: Andre Kurait <[email protected]>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1784 +/- ##
============================
============================
☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Description
This PR introduces delete operation support for delta snapshot migrations and significantly matures the RFS transformation schema to support a more complete set of bulk operation capabilities.
Key Changes:
DeltaMode.UPDATES_AND_DELETESis fully enabledDeltaMode.DELETES_ONLYis fully enabledBulkDocSectionstring-based approach with structured operation objectsIndexOpandDeleteOpimplementationsImpact:
This refactoring enables complete delta migration functionality while establishing a robust schema that can support advanced transformations and version control strategies during the migration process.
Issues Resolved
https://opensearch.atlassian.net/browse/MIGRATIONS-2582
Testing
Functional tests added for DeltaMode.UPDATES_AND_DELETES
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.