Skip to content

Comments

RFC-2083: Writer Sink API#2083

Merged
Xuanwo merged 4 commits intomainfrom
writer-sink
Apr 23, 2023
Merged

RFC-2083: Writer Sink API#2083
Xuanwo merged 4 commits intomainfrom
writer-sink

Conversation

@Xuanwo
Copy link
Member

@Xuanwo Xuanwo commented Apr 23, 2023

OpenDAL does not support streaming data uploads. Users must first load the data into memory and then send it to the writer.

let bs = balabala();
w.write(bs).await?;
let bs = daladala();
w.write(bs).await?;
...
w.close().await?;

There are two main drawbacks to OpenDAL:

To address this issue, it would be beneficial for OpenDAL to provide an API that allows users to pass a stream or reader directly into the writer.

Signed-off-by: Xuanwo <[email protected]>
@Xuanwo Xuanwo changed the title rfc: Add writer sink api RFC-2983: Writer Sink API Apr 23, 2023
@Xuanwo Xuanwo changed the title RFC-2983: Writer Sink API RFC-2083: Writer Sink API Apr 23, 2023
Copy link
Member

@PsiACE PsiACE left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

love this idea

Signed-off-by: Xuanwo <[email protected]>
@Xuanwo Xuanwo merged commit b5610e4 into main Apr 23, 2023
@Xuanwo Xuanwo deleted the writer-sink branch April 23, 2023 09:44
@suyanhanx
Copy link
Member

Great idea.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants