-
Notifications
You must be signed in to change notification settings - Fork 136
Description
At the time of the opening of this issue, Brim is at commit 9d0a742.
Since the backend already supports it, like we did for ZSON with #1486, it should be trivial to add Parquet as an Export format. However, as I was about to impulsively do this, I remembered that the backend still has this limitation:
$ zq -version
Version: v1.2.0-130-g8dd27d98
$ zq -f parquet 'fuse' ./src/test/shared/data/sample.tsv
union type unsupported
Since union types often show up in Zed formats (particularly as a side effect of things like applying fuse) my inclination is to wait on an improvement in the backend before we add it as an Export option.
For now, I've got a branch https://github.com/brimdata/brim/tree/parquet-export where I've got it working for the happy path. Separately I opened up brimdata/super#4215 to track what I'm observing in the API interactions that are currently preventing the app from reporting the failure scenarios to the user.