Skip to content

Tracking issues of QueryPairsWriter refactor #5976

@Xuanwo

Description

@Xuanwo

In opendal's code, there are many places need to build query pairs:

let mut url = format!("{}?list-type=2", self.endpoint);
if !p.is_empty() {
write!(url, "&prefix={}", percent_encode_path(&p))
.expect("write into string must succeed");
}
if !delimiter.is_empty() {
write!(url, "&delimiter={delimiter}").expect("write into string must succeed");
}
if let Some(limit) = limit {
write!(url, "&max-keys={limit}").expect("write into string must succeed");
}
if let Some(start_after) = start_after {
write!(url, "&start-after={}", percent_encode_path(&start_after))
.expect("write into string must succeed");
}
if !continuation_token.is_empty() {
// AWS S3 could return continuation-token that contains `=`
// which could lead `reqsign` parse query wrongly.
// URL encode continuation-token before starting signing so that
// our signer will not be confused.
write!(
url,
"&continuation-token={}",
percent_encode_path(continuation_token)
)
.expect("write into string must succeed");
}

However, many of them are poorly implemented, require unnecessary error handling, or consume additional memory.

This refactor intended to use newly added QueryPairsWriter to replace them all:

- let mut url = format!("{}?list-type=2", self.endpoint);
- if !p.is_empty() {
-     write!(url, "&prefix={}", percent_encode_path(&p))
-         .expect("write into string must succeed");
- }
+ let mut url = QueryPairsWriter::new(&self.endpoint);
+ url = url.push("list-type", "2");
+ if !p.is_empty() {
+     url = url.push("prefix", &percent_encode_path(&p));
+ }

Take #5977 as a full example. We don't need to replace every query build—only those with more than three query pairs, such as the list object operations. Please only migrate list operation.

Tasks

  • s3
  • aliyun_drive
  • azblob
  • azdls
  • azfile
  • b2
  • cos
  • gcs
  • gdrive
  • obs
  • onedrive
  • oss
  • swift

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions