Add ObjectStore::put_opts / Conditional Put (#4879) #4984
Merged
tustvold merged 12 commits intoapache:masterfrom Oct 27, 2023
Merged
Add ObjectStore::put_opts / Conditional Put (#4879) #4984tustvold merged 12 commits intoapache:masterfrom
tustvold merged 12 commits intoapache:masterfrom
Conversation
tustvold
commented
Oct 24, 2023
object_store/src/lib.rs
Outdated
Comment on lines
+860
to
+871
Contributor
Author
There was a problem hiding this comment.
This is kind of annoying, but is necessary because GCS doesn't support etag-based preconditions on put, only ifGenerationMatch
tustvold
force-pushed
the
conditional-put
branch
from
deb80bd to
f88574b
Compare
tustvold
force-pushed
the
conditional-put
branch
from
7617bad to
be34993
Compare
tustvold
commented
Oct 25, 2023
| let get_opts = storage.get_opts(&path, options).await.unwrap(); | ||
| let old = get_opts.bytes().await.unwrap(); | ||
| assert_eq!(old, b"foo".as_slice()); | ||
| assert_eq!(old, b"test".as_slice()); |
tustvold
commented
Oct 25, 2023
Comment on lines
+445
to
+447
| if let Some(version) = &options.version { | ||
| request = request.query(&[("generation", version)]); | ||
| } |
tustvold
commented
Oct 25, 2023
| } | ||
| } | ||
|
|
||
| #[derive(serde::Deserialize, Debug)] |
Contributor
Author
There was a problem hiding this comment.
These are moved to be shared with S3 (as they are the same)
tustvold
commented
Oct 25, 2023
| })?; | ||
| .context(CompleteMultipartRequestSnafu)?; | ||
|
|
||
| let etag = get_etag(result.headers()).context(MetadataSnafu)?; |
Contributor
Author
There was a problem hiding this comment.
This was a mistake introduced in #4971, this needs to extract the etag from the XML payload
tustvold
commented
Oct 25, 2023
| } | ||
| } | ||
|
|
||
| #[derive(Debug, Deserialize)] |
Contributor
Author
There was a problem hiding this comment.
These are moved to be shared with GCS
tustvold
commented
Oct 25, 2023
| async fn put(&self, location: &Path, bytes: Bytes) -> Result<PutResult> { | ||
| async fn put_opts(&self, location: &Path, bytes: Bytes, opts: PutOptions) -> Result<PutResult> { | ||
| if opts.mode != PutMode::Overwrite { | ||
| // TODO: Add support for If header - https://datatracker.ietf.org/doc/html/rfc2518#section-9.4 |
Contributor
Author
There was a problem hiding this comment.
I couldn't actually find an easy to run WebDav implementation that respected this header, I don't have a use-case for this currently so punted on this
tustvold
commented
Oct 25, 2023
| async fn put(&self, location: &Path, bytes: Bytes) -> Result<PutResult> { | ||
| async fn put_opts(&self, location: &Path, bytes: Bytes, opts: PutOptions) -> Result<PutResult> { | ||
| if matches!(opts.mode, PutMode::Update(_)) { | ||
| return Err(crate::Error::NotImplemented); |
Contributor
Author
There was a problem hiding this comment.
I intend to add support for this as a follow up PR
Merged
crepererum
approved these changes
Oct 27, 2023
2 tasks
tustvold
changed the title
6 tasks
alamb
pushed a commit
to alamb/arrow-rs
that referenced
this pull request
Mar 20, 2025
* Add version to PutResult * Conditional Put (#4879) * Don't support HttpStore * Add R2 Support * Update Azure StatusCode * Fixes * Clippy * Clippy * PutRequestBuilder * Clippy * Add stress test * Clippy
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes apache/arrow-rs-object-store#129
Relates to apache/arrow-rs-object-store#145
Rationale for this change
Allows optimistic concurrency control based transactions against object storage
What changes are included in this PR?
Are there any user-facing changes?