Repository navigation
[C++][Dataset][Parquet] Support parquet modular encryption in the new Dataset API #29238
Description
In order for the new Dataset API to fully support PME, the same writer properties that include file_encryption_properties shouldn’t be used for the whole dataset. file_encryption_properties should be per file, for example in order to support key rotation ARROW-9960 .
Design document: https://docs.google.com/document/d/13EysCNC6-Nu9wnJ8YpdzmD-aMLn4i2KXUJTNqIihy7A/edit#
Reporter: Maya Anderson / @andersonm-ibm
Related issues:
- [C++][Parquet] Enable external material and rotation for encryption keys #25986 (Dependency)
- [Python] decryption_properties missing in pyarrow.parquet.ParquetDataset #32564 (supercedes)
- [Python] Expose decryption properties in ParquetFragmentScanOptions #31410 (is depended upon by)
PRs and other links:
Note: This issue was originally created as ARROW-13593. Please see the migration documentation for further details.
Activity
asfimport commentedon Apr 21, 2022
Krisztian Szucs / @kszucs:
Postponing to 9.0
asfimport commentedon Jul 15, 2022
Neal Richardson / @nealrichardson:
This issue has been inactive for 3 months, so it has been unassigned and marked as unstarted. If you are still working on this, feel free to reassign yourself and resume progress.
raulcd commentedon Oct 10, 2023
@ianmcook @jorisvandenbossche this doesn't seem like a blocker, right?
ianmcook commentedon Oct 10, 2023
@raulcd this one is very close to being finished and we would like to get it into 14.0.0 if possible
GH-29238 [C++][Dataset][Parquet] Support parquet modular encryption i…
jorisvandenbossche commentedon Oct 11, 2023
@tolleybot if you could answer here on this comment, I can assign the issue to you (some strange limitation of github that this is required .. ;))
apacheGH-29238 [C++][Dataset][Parquet] Support parquet modular encryp…
apacheGH-29238 [C++][Dataset][Parquet] Support parquet modular encryp…