Skip to content

Comments

services/cos: Implement Write Returns Metadata for cos#5777

Merged
Xuanwo merged 3 commits intoapache:mainfrom
Ziy1-Tan:md-cos
Mar 19, 2025
Merged

services/cos: Implement Write Returns Metadata for cos#5777
Xuanwo merged 3 commits intoapache:mainfrom
Ziy1-Tan:md-cos

Conversation

@Ziy1-Tan
Copy link
Contributor

@Ziy1-Tan Ziy1-Tan commented Mar 15, 2025

Which issue does this PR close?

Part of #5557 .

Test result of OPENDAL_COS_ENABLE_VERSIONING=true OPENDAL_TEST=cos cargo test behavior::test_write --features tests

image

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

@Ziy1-Tan Ziy1-Tan requested a review from Xuanwo as a code owner March 15, 2025 09:24
@Ziy1-Tan
Copy link
Contributor Author

Hey, @meteorgan @Xuanwo! The behavior tests of metadata have been fixed, Would you have to take a look? :)

@meteorgan
Copy link
Contributor

There shouldn't be any failed behavior tests. If a feature hasn't been implemented yet, the corresponding test should not be executed. we have some conditions to control this. for example:

pub async fn test_write_with_if_not_exists(op: Operator) -> Result<()> {
    if !op.info().full_capability().write_with_if_not_exists {
        return Ok(());
    }


let resp = self.core.send(req).await?;

let meta = Self::parse_metadata(resp.headers())?;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In #5710 (comment), we found that cos may return null as the version id which could cause test_write_with_append_returns_metadata to fail. However this issue hasn't been addressed here. Did i miss something ? Also, when running behavior tests, did you enable OPENDAL_COS_ENABLE_VERSIONING ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will cover it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer we skip the version id when it's null. what do you think ? @Xuanwo

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

.await?;

let result: CompleteMultipartUploadResult =
quick_xml::de::from_reader(resp.body().clone().reader())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can avoid this clone by moving resp.status() before this line.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resp.body() is a shared reference. Clone or mutable reference will work.

Comment on lines 228 to 230
// COS return null as the version id when it's an appendable object.
// Refer to https://cloud.tencent.com/document/product/436/7741
meta.set_version("null");
Copy link
Contributor

@meteorgan meteorgan Mar 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should've made it clear: we don't need to return null as version id since it doesn't make sense to users. If parsing version id returns in null, we simply shouldn't set it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the stat method in backend.rs may have the same issue.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

Copy link
Contributor

@meteorgan meteorgan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. We could try to merge this PR now. Thanks very much @Ziy1-Tan

@Xuanwo
Copy link
Member

Xuanwo commented Mar 19, 2025

Thank you @Ziy1-Tan for the great work and thank you @meteorgan for the review!

@Xuanwo Xuanwo merged commit 4c8153d into apache:main Mar 19, 2025
92 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants