Improve documentation on writing parquet, including multiple threads #7321
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
Author
|
I have a few more things to write but ran out of time today |
alamb
force-pushed
the
alamb/parallel-docs
branch
from
6a8012a to
9026cc0
Compare
alamb
changed the title
alamb
force-pushed
the
alamb/parallel-docs
branch
from
9026cc0 to
1e5e072
Compare
alamb
commented
Mar 23, 2025
Contributor
Author
|
@etseidl I wonder if I could trouble you for a review of this PR? |
etseidl
approved these changes
Mar 25, 2025
parquet/src/lib.rs
Outdated
Comment on lines
59
to
60
| //! to leverage the wide range of data transforms provided by the [arrow] crate, the | ||
| //! ecosystem of [Arrow] compatible systems. |
Contributor
There was a problem hiding this comment.
Not sure why you dropped "and by" here.
Contributor
Author
There was a problem hiding this comment.
I think I was exercising my inner copy editor and trying to reduce the number of words being used to express this concept. I agree and by was a unintended casualty. I will put it back
alamb
commented
Mar 25, 2025
Co-authored-by: Ed Seidl <[email protected]>
Contributor
Author
|
Thanks again @etseidl |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Rationale for this change
I spent a while looking for how to write parquet data with multiple threads and I knew the functionality exists
Also I always get confused looking at the parquet documentation as there are similarly named structures and in different modules.
What changes are included in this PR?
Let's leave some more links in the docs to make it easier to APIs related to parallelism
Also, mention in the key structures crate level docs
Are there any user-facing changes?
Documentation only.
There are no functional changes