Skip to content

Commit 4d1197e

Browse files
committed
Nits and try to fix AppVeyor link errors
1 parent be154fd commit 4d1197e

5 files changed

Lines changed: 42 additions & 36 deletions

File tree

‎docs/source/developers/python.rst‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -350,8 +350,9 @@ Now, build pyarrow:
350350
If you did build one of the optional components (in C++), you need to set the
351351
corresponding ``PYARROW_WITH_$COMPONENT`` environment variable to 1.
352352

353-
If you built with ``PARQUET_REQUIRE_ENCRYPTION`` (in C++), you need to set the
354-
corresponding ``PYARROW_WITH_PARQUET_ENCRYPTION`` environment variable to 1.
353+
Similarly, if you built with ``PARQUET_REQUIRE_ENCRYPTION`` (in C++), you
354+
need to set the corresponding ``PYARROW_WITH_PARQUET_ENCRYPTION`` environment
355+
variable to 1.
355356

356357
If you wish to delete pyarrow build before rebuilding navigate to the ``arrow/python``
357358
folder and run ``git clean -Xfd .``.

‎docs/source/python/parquet.rst‎

Lines changed: 32 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -635,8 +635,8 @@ Reading an encrypted Parquet file:
635635
636636
637637
In order to create the encryption and decryption properties, a
638-
:class:`~pyarrow.parquet.CryptoFactory` should be created and initialized with
639-
KMS Client details, as described below.
638+
:class:`pyarrow.parquet_encryption.CryptoFactory` should be created and
639+
initialized with KMS Client details, as described below.
640640

641641

642642
KMS Client
@@ -646,15 +646,17 @@ The master encryption keys should be kept and managed in a production-grade
646646
Key Management System (KMS), deployed in the user's organization. Using Parquet
647647
encryption requires implementation of a client class for the KMS server.
648648
Any KmsClient implementation should implement the informal interface
649-
defined by :class:`~pyarrow.parquet.KmsClient` as following:
649+
defined by :class:`pyarrow.parquet_encryption.KmsClient` as following:
650650

651651
.. code-block:: python
652652
653+
import pyarrow.parquet_encryption as pe
654+
655+
class MyKmsClient(pe.KmsClient):
653656
654-
class MyKmsClient(pq.KmsClient):
655657
"""An example KmsClient implementation skeleton"""
656658
def __init__(self, kms_connection_configuration):
657-
pq.KmsClient.__init__(self)
659+
pe.KmsClient.__init__(self)
658660
# Any KMS-specific initialization based on
659661
# kms_connection_configuration comes here
660662
@@ -668,10 +670,11 @@ defined by :class:`~pyarrow.parquet.KmsClient` as following:
668670
# master_key_identifier
669671
return key_bytes
670672
671-
The concrete implementation will be loaded at runtime by a factory method
672-
provided by the user. This factory method will be used to initialize the
673-
:class:`~pyarrow.parquet.CryptoFactory` for creating file encryption and
674-
decryption properties.
673+
The concrete implementation will be loaded at runtime by a factory function
674+
provided by the user. This factory function will be used to initialize the
675+
:class:`pyarrow.parquet_encryption.CryptoFactory` for creating file encryption
676+
and decryption properties.
677+
675678
For example, in order to use the ``MyKmsClient`` defined above:
676679

677680
.. code-block:: python
@@ -694,7 +697,7 @@ above.
694697
KMS connection configuration
695698
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
696699

697-
Configuration of connection to KMS (:class:`~pyarrow.parquet.KmsConnectionConfig`
700+
Configuration of connection to KMS (:class:`pyarrow.parquet_encryption.KmsConnectionConfig`
698701
used when creating file encryption and decryption properties) includes the
699702
following options:
700703

@@ -707,37 +710,36 @@ following options:
707710
Encryption configuration
708711
~~~~~~~~~~~~~~~~~~~~~~~~
709712

710-
:class:`~pyarrow.parquet.EncryptionConfiguration` (used when creating file
711-
encryption properties) includes the following options:
713+
:class:`pyarrow.parquet_encryption.EncryptionConfiguration` (used when
714+
creating file encryption properties) includes the following options:
712715

713-
* ``footer_key``, ID of the master key for footer encryption/signing.
716+
* ``footer_key``, the ID of the master key for footer encryption/signing.
714717
* ``column_keys``, which columns to encrypt with which key. Dictionary with
715718
master key IDs as the keys, and column name lists as the values,
716719
e.g. ``{key1: [col1, col2], key2: [col3]}`` .
717720
* ``encryption_algorithm``, the Parquet encryption algorithm.
718-
Can be ``AES_GCM_V1`` (default), or ``AES_GCM_CTR_V1``.
721+
Can be ``AES_GCM_V1`` (default) or ``AES_GCM_CTR_V1``.
719722
* ``plaintext_footer``, whether to write the file footer in plain text (otherwise it is encrypted).
720723
* ``double_wrapping``, whether to use double wrapping - where data encryption keys (DEKs)
721724
are encrypted with key encryption keys (KEKs), which in turn are encrypted
722-
with master keys. If set to ``false``, single wrapping is used - where DEKs are
723-
encrypted directly with master keys.
724-
* ``cache_lifetime``, lifetime of cached entities (key encryption keys,
725-
local wrapping keys, KMS client objects). Type - ``datetime.timedelta``.
725+
with master encryption keys (MEKs). If set to ``false``, single wrapping is
726+
used - where DEKs are encrypted directly with MEKs.
727+
* ``cache_lifetime``, the lifetime of cached entities (key encryption keys,
728+
local wrapping keys, KMS client objects) represented as a ``datetime.timedelta``.
726729
* ``internal_key_material``, whether to store key material inside Parquet file footers;
727730
this mode doesn’t produce additional files. If set to ``false``, key material is
728731
stored in separate files in the same folder, which enables key rotation for
729732
immutable Parquet files.
730-
* ``data_key_length_bits``, length of data encryption keys (DEKs), randomly
733+
* ``data_key_length_bits``, the length of data encryption keys (DEKs), randomly
731734
generated by Parquet key management tools. Can be 128, 192 or 256 bits.
732735

733736
.. note::
734-
By default, Parquet implements a "double envelope encryption" mode, that
735-
minimizes the interaction of the program with a KMS server. In this mode,
736-
the DEKs are encrypted with "key encryption keys" (KEKs, randomly generated
737-
by Parquet). The KEKs are encrypted with MEKs in KMS; the result and the
738-
KEK itself are cached in the process memory. Users interested in regular
739-
envelope encryption, can switch to it by setting the ``double_wrapping``
740-
parameter of :class:`~pyarrow.parquet.EncryptionConfiguration` to false.
737+
When ``double_wrapping`` is true, Parquet implements a "double envelope
738+
encryption" mode that minimizes the interaction of the program with a KMS
739+
server. In this mode, the DEKs are encrypted with "key encryption keys"
740+
(KEKs, randomly generated by Parquet). The KEKs are encrypted with "master
741+
encryption keys" (MEKs) in the KMS; the result and the KEK itself are
742+
cached in the process memory.
741743

742744
An example encryption configuration:
743745

@@ -753,9 +755,8 @@ An example encryption configuration:
753755
Decryption configuration
754756
~~~~~~~~~~~~~~~~~~~~~~~~
755757

756-
:class:`~pyarrow.parquet.DecryptionConfiguration` (used when creating file
757-
decryption properties) is optional and it includes the following options:
758-
759-
* ``cache_lifetime``, lifetime of cached entities (key encryption keys, local
760-
wrapping keys, KMS client objects). Type - ``datetime.timedelta``.
758+
:class:`pyarrow.parquet_encryption.DecryptionConfiguration` (used when creating
759+
file decryption properties) is optional and it includes the following options:
761760

761+
* ``cache_lifetime``, the lifetime of cached entities (key encryption keys, local
762+
wrapping keys, KMS client objects) represented as a ``datetime.timedelta``.

‎python/CMakeLists.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -640,6 +640,7 @@ endif()
640640

641641
if(PYARROW_BUILD_PARQUET)
642642
target_link_libraries(_parquet PRIVATE ${PARQUET_LINK_LIBS})
643+
target_link_libraries(_parquet_encryption PRIVATE ${PARQUET_LINK_LIBS})
643644
endif()
644645

645646
if(PYARROW_BUILD_PLASMA)

python/pyarrow/tests/parquet/common_parquet_encryption.py renamed to python/pyarrow/tests/parquet/encryption.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,4 +57,4 @@ def verify_file_encrypted(path):
5757
with open(path, "rb") as file:
5858
magic_str = file.read(4)
5959
# Verify magic string for parquet with encrypted footer is PARE
60-
assert(magic_str == b'PARE')
60+
assert magic_str == b'PARE'

python/pyarrow/tests/parquet/test_parquet_encryption.py renamed to python/pyarrow/tests/parquet/test_encryption.py

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,11 +21,13 @@
2121
try:
2222
import pyarrow.parquet as pq
2323
import pyarrow.parquet_encryption as pe
24-
from common_parquet_encryption import InMemoryKmsClient
25-
from common_parquet_encryption import verify_file_encrypted
2624
except ImportError:
2725
pq = None
2826
pe = None
27+
else:
28+
from pyarrow.tests.parquet.encryption import (
29+
InMemoryKmsClient, verify_file_encrypted)
30+
2931

3032
PARQUET_NAME = 'encrypted_table.in_mem.parquet'
3133
FOOTER_KEY = b"0123456789112345"
@@ -38,6 +40,7 @@
3840
# Ignore these with pytest ... -m 'not parquet_encryption'
3941
pytestmark = pytest.mark.parquet_encryption
4042

43+
4144
@pytest.fixture(scope='module')
4245
def data_table():
4346
data_table = pa.Table.from_pydict({

0 commit comments

Comments
 (0)