Skip to content

zio/anyio: auto-detect Parquet and VNG#4270

Merged
nwt merged 5 commits intomainfrom
detect-parquet-and-vng
Dec 23, 2022
Merged

zio/anyio: auto-detect Parquet and VNG#4270
nwt merged 5 commits intomainfrom
detect-parquet-and-vng

Conversation

@nwt
Copy link
Member

@nwt nwt commented Dec 14, 2022

Both formats require a seekable reader so this only works with files and URIs that are not gzip-compressed. It does not work with standard input, fifos, or gzip-compressed data.

Closes #2517.

Both formats require a seekable reader so this only works with files and
URIs that are not gzip-compressed.  It does not work with standard
input, fifos, or gzip-compressed data.

Closes #2517.
@nwt nwt requested a review from a team December 14, 2022 21:58
parquetErr = fmt.Errorf("parquet: %w", parquetErr)
vngErr = fmt.Errorf("vng: %w", vngErr)
} else {
parquetErr = errors.New("parquet: reader cannot seek")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"reader cannot seek" is describing the implementation not the supported types of inputs begging the question "what's a reader?". How about seekable input required for auto-detection (consider "-i parquet")

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mccanne: Good point, though "consider '-i parquet'" isn't quite the right advice since that won't make the input seekable, and that's the requirement here. I'll go with "seekable input required for auto-detection" unless you've got other ideas.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in f82fb69, though the exact language I used is "auto-detection requires seekable input" to align with lineErr := errors.New("line: auto-detection not supported").

@nwt nwt requested review from a team and mccanne December 19, 2022 19:47
@nwt nwt merged commit 43bc182 into main Dec 23, 2022
@nwt nwt deleted the detect-parquet-and-vng branch December 23, 2022 20:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Auto-detect all input formats for files

3 participants