PR#2: Assorted fixes and improvements, especially to the new python3-pyarrow subpackage - rpms/libarrow

		`@@ -0,0 +1,31 @@`
		`+ From 71a55fb0f72c5bba26ac77f841a716d00f149c28 Mon Sep 17 00:00:00 2001`
		`+ From: Miles Granger <[email protected]>`
		`+ Date: Wed, 17 Aug 2022 13:15:45 +0200`
		`+ Subject: [PATCH] Exclude Python conftest and tests when PYARROW_INSTALL_TESTS`
		`+ not on`
		`+`
		`+ ---`
		`+ python/setup.py \| 6 ++++--`
		`+ 1 file changed, 4 insertions(+), 2 deletions(-)`
		`+`
		`+ diff --git a/python/setup.py b/python/setup.py`
		`+ index 97e0f07720bb..1d03a47ee42b 100755`
		`+ --- a/python/setup.py`
		`+ +++ b/python/setup.py`
		`+ @@ -623,12 +623,14 @@ def has_ext_modules(foo):`
		`+ if strtobool(os.environ.get('PYARROW_INSTALL_TESTS', '1')):`
		`+ packages = find_namespace_packages(include=['pyarrow*'])`
		`+ else:`
		`+ - packages = find_namespace_packages(include=['pyarrow*'],`
		`+ - exclude=["pyarrow.tests*"])`
		`+ + packages = find_namespace_packages(`
		`+ + include=["pyarrow*"],`
		`+ + exclude=["pyarrow.tests*", "pyarrow.conftest"])`
		`+`
		`+`
		`+ setup(`
		`+ name='pyarrow',`
		`+ + package_dir={"": "."},`
		`+ packages=packages,`
		`+ zip_safe=False,`
		`+ package_data={'pyarrow': ['.pxd', '.pyx', 'includes/*.pxd']},`

libarrow.spec

file modified

+50 -13

		`@@ -31,12 +31,15 @@`

		`Name: libarrow`
		`Version: 9.0.0`
		`- Release: 2%{?dist}`
		`+ Release: 3%{?dist}`
		`Summary: A toolbox for accelerated data interchange and in-memory processing`
		`License: ASL 2.0`
		`URL: https://arrow.apache.org/`
		`Requires: %{name}-doc = %{version}-%{release}`
		`Source0: https://dist.apache.org/repos/dist/release/arrow/arrow-%{version}/apache-arrow-%{version}.tar.gz`
		`+ # ARROW-17389: [Python] Exclude conftest and tests when PYARROW_INSTALL_TESTS=0`
		`+ # Fixes https://issues.apache.org/jira/browse/ARROW-17389`
		`+ Patch: https://github.com/apache/arrow/pull/13904.patch`
		`# Apache ORC (liborc) has numerous compile errors and apparently assumes`
		`# a 64-bit build and runtime environment. This is only consumer of the liborc`
		`# package, and in turn the only consumer of this and liborc is Ceph, which`
		`@@ -89,12 +92,8 @@`
		`BuildRequires: gobject-introspection-devel`
		`BuildRequires: gtk-doc`

		`- # Additional pyarrow build requirements`
		`- BuildRequires: python3-pip`
		`- BuildRequires: python3-setuptools`
		`+ # Additional pyarrow build requirements; see also %%generate_buildrequires`
		`BuildRequires: python3dist(cffi)`
		`- BuildRequires: python3dist(cython)`
		`- BuildRequires: python3dist(wheel)`

		`%description`
		`Apache Arrow defines a language-independent columnar memory`
		`@@ -707,10 +706,8 @@`
		`%description -n python3-pyarrow`
		`Python library for Apache Arrow`

		`- %files -n python3-pyarrow`
		`+ %files -n python3-pyarrow -f %{pyproject_files}`
		`%{_bindir}/plasma_store`
		`- %dir %{python3_sitearch}/pyarrow`
		`- %{python3_sitearch}/pyarrow/*.so`
		`%exclude %{python3_sitearch}/pyarrow/lib_api.h`
		`%exclude %{python3_sitearch}/pyarrow/include`

		`@@ -719,17 +716,26 @@`
		`%package -n python3-pyarrow-devel`
		`Summary: Development files for python3-pyarrow`

		`+ Requires: python3-pyarrow%{?_isa} = %{version}-%{release}`
		`+`
		`%description -n python3-pyarrow-devel`
		`Development files for python3-pyarrow`

		`- %files -n python3-pyarrow-devel -f %{_pyproject_ghost_distinfo}`
		`- %exclude %{python3_sitearch}/pyarrow/*.so`
		`- %{python3_sitearch}/pyarrow/*`
		`+ %files -n python3-pyarrow-devel`
		`+ %{python3_sitearch}/pyarrow/lib_api.h`
		`+ %{python3_sitearch}/pyarrow/include`

		`#--------------------------------------------------------------------`

		`%prep`
		`%autosetup -p1 -n apache-arrow-%{version}`
		`+ # We do not need to (nor can we) build for an old version of numpy:`
		`+ sed -r -i 's/(oldest-supported-)(numpy)/\2/' python/pyproject.toml`
		`+`
		`+ %generate_buildrequires`
		`+ pushd python >/dev/null`
		`+ %pyproject_buildrequires`
		`+ popd >/dev/null`

		`%build`
		`pushd cpp`
		`@@ -750,6 +756,7 @@`
		`%endif`
		`-DARROW_PYTHON:BOOL=ON \`
		`-DARROW_JEMALLOC:BOOL=OFF \`
		`+ -DARROW_SIMD_LEVEL:STRING='NONE' \`
		`-DGRPC_SOURCE="SYSTEM" \`
		`-Dxsimd_SOURCE="SYSTEM" \`
		`%if %{with use_s3}`
		`@@ -761,7 +768,7 @@`
		`-DARROW_WITH_SNAPPY:BOOL=ON \`
		`-DARROW_WITH_ZLIB:BOOL=ON \`
		`-DARROW_WITH_ZSTD:BOOL=ON \`
		`- -DARROW_WITH_XSIMD:BOOL=ON \`
		`+ -DARROW_USE_XSIMD:BOOL=ON \`
		`-DARROW_BUILD_STATIC:BOOL=OFF \`
		`-DCMAKE_BUILD_TYPE=RelWithDebInfo \`
		`-DCMAKE_COLOR_MAKEFILE:BOOL=OFF \`
		`@@ -802,6 +809,9 @@`
		`PYARROW_WITH_DATASET=1 \`
		`PYARROW_WITH_FLIGHT=1 \`
		`PYARROW_WITH_PARQUET=1 \`
		`+ %{?with_use_plasma:PYARROW_WITH_PLASMA=1} \`
		`+ PYARROW_WITH_PARQUET_ENCRYPTION=1 \`
		`+ %{?with_use_gandiva:PYARROW_WITH_GANDIVA=1} \`
		`PYARROW_PARALLEL=%{_smp_build_ncpus} \`
		`PYARROW_INSTALL_TESTS=0`
		`%pyproject_wheel`
		`@@ -813,7 +823,9 @@`
		`%install`

		`pushd python`
		`+ export PYARROW_INSTALL_TESTS=0`
		`%pyproject_install`
		`+ %pyproject_save_files pyarrow`
		`popd`

		`pushd c_glib`
		`@@ -824,9 +836,34 @@`
		`%cmake_install`
		`popd`

		`+`
		`+ %check`
		`+ export LD_LIBRARY_PATH='%{buildroot}%{_libdir}'`
		`+ # While https://github.com/apache/arrow/pull/13904 partially fixes`
		`+ # https://issues.apache.org/jira/browse/ARROW-17389, conftest.py is still`
		`+ # installed. We must skip testing it because it would import pytest.`
		`+ #`
		`+ # Additionally, skip subpackages corresponding to missing optional`
		`+ # functionality.`
		`+ %{pyproject_check_import \`
		`+ -e 'pyarrow.conftest' \`
		`+ -e 'pyarrow.orc' -e 'pyarrow._orc' \`
		`+ %{?!with_use_plasma:-e 'pyarrow.plasma' -e 'pyarrow._plasma'} \`
		`+ -e 'pyarrow.substrait' -e 'pyarrow._substrait' \`
		`+ -e 'pyarrow.cuda'}`
		`+`
		`#--------------------------------------------------------------------`

		`%changelog`
		`+ * Wed Aug 17 2022 Benjamin A. Beasley <[email protected]> - 9.0.0-3`
		`+ - Use %%pyproject_save_files to fix pyarrow metadata`
		`+ - Use generated BR’s for pyarrow`
		`+ - Add import-only “smoke tests” for pyarrow`
		`+ - Stop requiring SSE4.2 on x86-family platforms`
		`+ - Correct ARROW_WITH_XSIMD; should be ARROW_USE_XSIMD`
		`+ - Enable more optional functionality in pyarrow`
		`+ - Fix pyarrow tests installed despite PYARROW_INSTALL_TESTS=0`
		`+`
		`* Wed Aug 10 2022 Kaleb S. KEITHLEY <kkeithle [at] redhat.com> - 9.0.0-2`
		`- Arrow 9.0.0, enable python, i.e. python3-pyarrow, subpackage`

music commented 4 years ago

Please see the changelog message or commit messages for details. I’m happy to answer questions or adjust to your preferences.

Note that “Stop requiring SSE4.2 on x86-family platforms” and “Correct ARROW_WITH_XSIMD; should be ARROW_USE_XSIMD” affect the entire package, not just the subpackage for pyarrow.

It’s probably possible and desirable to run the actual upstream test suite for pyarrow, but I didn’t work on that.

music commented 4 years ago

$ rpm -q --provides -p results_libarrow/9.0.0/3.fc38/python3-pyarrow-9.0.0-3.fc37.x86_64.rpm 
lib.cpython-311-x86_64-linux-gnu.so()(64bit)
python-pyarrow = 9.0.0-3.fc37
python3-pyarrow = 9.0.0-3.fc37
python3-pyarrow(x86-64) = 9.0.0-3.fc37
python3.11-pyarrow = 9.0.0-3.fc37
python3.11dist(pyarrow) = 9
python3dist(pyarrow) = 9

1 new commit added