Skip to content

arrow-cpp: 9.0.0 -> 10.0.1#198805

Closed
cpcloud wants to merge 3 commits intoNixOS:masterfrom
cpcloud:arrow-10
Closed

arrow-cpp: 9.0.0 -> 10.0.1#198805
cpcloud wants to merge 3 commits intoNixOS:masterfrom
cpcloud:arrow-10

Conversation

@cpcloud
Copy link
Contributor

@cpcloud cpcloud commented Oct 31, 2022 •
edited
Loading

Description of changes

This PR bumps arrow-cpp to version 10.0.0.

Notes:

  1. This PR currently depends on google-cloud-cpp: 1.38.0 -> 2.4.0 #198469.
  2. To avoid what I believe to be cxx abi issues, I had to set cxxStandard to 17
    for abseil-cpp, and -DCMAKE_CXX_STANDARD=17 for grpc and google-cloud-cpp.     The default of 17 works!
  3. Arrow itself now requires CMAKE_CXX_STANDARD to be set to 17.

I'm not sure if this should go to staging, but happy to move it there if so.

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 22.11 Release Notes (or backporting 22.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

@github-actions github-actions bot added the 6.topic: python Python is a high-level, general-purpose programming language. label Oct 31, 2022
cpcloud
cpcloud commented Oct 31, 2022
View reviewed changes
@ofborg ofborg bot requested review from LnL7, marsam, tobim and veprbl October 31, 2022 16:50
@ofborg ofborg bot added 10.rebuild-darwin: 101-500 This PR causes between 101 and 500 packages to rebuild on Darwin. 10.rebuild-linux: 101-500 This PR causes between 101 and 500 packages to rebuild on Linux. labels Oct 31, 2022
tobim
tobim reviewed Nov 1, 2022
View reviewed changes
Copy link
Contributor

@tobim tobim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking care of the update again.

@ofborg ofborg bot requested a review from tobim November 1, 2022 11:02
@cpcloud cpcloud force-pushed the arrow-10 branch 2 times, most recently from f687d44 to 047060b Compare November 1, 2022 15:15
veprbl
veprbl reviewed Nov 1, 2022
View reviewed changes
veprbl
veprbl reviewed Nov 1, 2022
View reviewed changes
veprbl
veprbl reviewed Nov 1, 2022
View reviewed changes
@ofborg ofborg bot requested a review from veprbl November 1, 2022 17:30
@cpcloud
Copy link
Contributor Author

cpcloud commented Nov 2, 2022

It appears that arrow 10 doesn't require building with python 🎉

cpcloud
cpcloud commented Nov 2, 2022
View reviewed changes
cpcloud
cpcloud commented Nov 2, 2022
View reviewed changes
cpcloud
cpcloud commented Nov 2, 2022
View reviewed changes
@cpcloud
Copy link
Contributor Author

cpcloud commented Nov 2, 2022

@veprbl Can you give this another look when you get a chance?

@cpcloud cpcloud mentioned this pull request Nov 2, 2022
Closed
1 task
marsam
marsam reviewed Nov 2, 2022
View reviewed changes
@ofborg ofborg bot requested a review from marsam November 2, 2022 13:59
@ofborg ofborg bot removed the 10.rebuild-linux: 501-1000 This PR causes many rebuilds on Linux and should normally target the staging branches. label Dec 9, 2022
@veprbl
Copy link
Member

veprbl commented Dec 9, 2022 

Undefined symbols for architecture x86_64:
  "testing::Matcher<std::__1::basic_string_view<char, std::__1::char_traits<char> > const&>::Matcher(char const*)", referenced from:
      testing::internal::ElementsAreMatcherImpl<std::__1::vector<std::__1::basic_string_view<char, std::__1::char_traits<char> >, std::__1::allocator<std::__1::basic_string_view<char, std::__1::char_traits<char> > > > const&>::ElementsAreMatcherImpl<std::__1::__wrap_iter<char const* const*> >(std::__1::__wrap_iter<char const* const*>, std::__1::__wrap_iter<char const* const*>) in array_test.cc.o
      testing::internal::ElementsAreMatcherImpl<std::__1::vector<std::__1::basic_string_view<char, std::__1::char_traits<char> >, std::__1::allocator<std::__1::basic_string_view<char, std::__1::char_traits<char> > > > const&>::ElementsAreMatcherImpl<std::__1::__wrap_iter<char const* const*> >(std::__1::__wrap_iter<char const* const*>, std::__1::__wrap_iter<char const* const*>) in array_binary_test.cc.o
ld: symbol(s) not found for architecture x86_64
clang-11: error: linker command failed with exit code 1 (use -v to see invocation)

@cpcloud cpcloud force-pushed the arrow-10 branch 4 times, most recently from 09f1e25 to 5fb5cde Compare December 9, 2022 11:04
@cpcloud
Copy link
Contributor Author

cpcloud commented Dec 9, 2022 •
edited
Loading

@veprbl On what OS are you running that?

Separately, it looks like aarch64-linux and both of the darwins are not working, aarch64-linux is failing to build grpc on master which transitively prevents arrow-cpp from building.

@veprbl
Copy link
Member

veprbl commented Dec 10, 2022

@veprbl On what OS are you running that?

x86_64-darwin, but aarch64-darwin should see the same issue.

@veprbl
Copy link
Member

veprbl commented Dec 10, 2022

I think darwin failure would need to be reported upstream.

@cpcloud
Copy link
Contributor Author

cpcloud commented Dec 10, 2022

@veprbl I don't have access to macOS, would you mind reporting upstream and ccing me?

@cpcloud cpcloud force-pushed the arrow-10 branch 3 times, most recently from 2913e1a to 9d837d3 Compare December 17, 2022 13:08
@veprbl veprbl mentioned this pull request Dec 22, 2022
Merged
13 tasks
@tobim
Copy link
Contributor

tobim commented Dec 28, 2022

pyarrow has a failing test on darwin:

python3.10-pyarrow> =================================== FAILURES ===================================
python3.10-pyarrow> _______________________________ test_cython_api ________________________________
python3.10-pyarrow> tmpdir = local('/private/tmp/nix-build-python3.10-pyarrow-10.0.1.drv-1/pytest-of-_nixbld1/pytest-0/test_cython_api0')
python3.10-pyarrow>     @pytest.mark.cython
python3.10-pyarrow>     def test_cython_api(tmpdir):
python3.10-pyarrow>         """
python3.10-pyarrow>         Basic test for the Cython API.
python3.10-pyarrow>         """
python3.10-pyarrow>         # Fail early if cython is not found
python3.10-pyarrow>         import cython  # noqa
python3.10-pyarrow>         with tmpdir.as_cwd():
python3.10-pyarrow>             # Set up temporary workspace
python3.10-pyarrow>             pyx_file = 'pyarrow_cython_example.pyx'
python3.10-pyarrow>             shutil.copyfile(os.path.join(here, pyx_file),
python3.10-pyarrow>                             os.path.join(str(tmpdir), pyx_file))
python3.10-pyarrow>             # Create setup.py file
python3.10-pyarrow>             setup_code = setup_template.format(pyx_file=pyx_file,
python3.10-pyarrow>                                                compiler_opts=compiler_opts,
python3.10-pyarrow>                                                test_ld_path=test_ld_path)
python3.10-pyarrow>             with open('setup.py', 'w') as f:
python3.10-pyarrow>                 f.write(setup_code)
python3.10-pyarrow>             # ARROW-2263: Make environment with this pyarrow/ package first on the
python3.10-pyarrow>             # PYTHONPATH, for local dev environments
python3.10-pyarrow>             subprocess_env = test_util.get_modified_env_with_pythonpath()
python3.10-pyarrow>             # Compile extension module
python3.10-pyarrow>             subprocess.check_call([sys.executable, 'setup.py',
python3.10-pyarrow>                                    'build_ext', '--inplace'],
python3.10-pyarrow>                                   env=subprocess_env)
python3.10-pyarrow>             # Check basic functionality
python3.10-pyarrow>             orig_path = sys.path[:]
python3.10-pyarrow>             sys.path.insert(0, str(tmpdir))
python3.10-pyarrow>             try:
python3.10-pyarrow>                 mod = __import__('pyarrow_cython_example')
python3.10-pyarrow>                 check_cython_example_module(mod)
python3.10-pyarrow>             finally:
python3.10-pyarrow>                 sys.path = orig_path
python3.10-pyarrow>             # Check the extension module is loadable from a subprocess without
python3.10-pyarrow>             # pyarrow imported first.
python3.10-pyarrow>             code = """if 1:
python3.10-pyarrow>                 import sys
python3.10-pyarrow>                 import os
python3.10-pyarrow>                 try:
python3.10-pyarrow>                     # Add dll directory was added on python 3.8
python3.10-pyarrow>                     # and is required in order to find extra DLLs
python3.10-pyarrow>                     # only for win32
python3.10-pyarrow>                     for dir in {library_dirs}:
python3.10-pyarrow>                         os.add_dll_directory(dir)
python3.10-pyarrow>                 except AttributeError:
python3.10-pyarrow>                     pass
python3.10-pyarrow>                 mod = __import__({mod_name!r})
python3.10-pyarrow>                 arr = mod.make_null_array(5)
python3.10-pyarrow>                 assert mod.get_array_length(arr) == 5
python3.10-pyarrow>                 assert arr.null_count == 5
python3.10-pyarrow>             """.format(mod_name='pyarrow_cython_example',
python3.10-pyarrow>                        library_dirs=pa.get_library_dirs())
python3.10-pyarrow>             var = None
python3.10-pyarrow>             if sys.platform == 'win32':
python3.10-pyarrow>                 if not hasattr(os, 'add_dll_directory'):
python3.10-pyarrow>                     # Python 3.8 onwards don't check extension module DLLs on path
python3.10-pyarrow>                     # we have to use os.add_dll_directory instead.
python3.10-pyarrow>                     delim, var = ';', 'PATH'
python3.10-pyarrow>             else:
python3.10-pyarrow>                 delim, var = ':', 'LD_LIBRARY_PATH'
python3.10-pyarrow>             if var:
python3.10-pyarrow>                 subprocess_env[var] = delim.join(
python3.10-pyarrow>                     pa.get_library_dirs() + [subprocess_env.get(var, '')]
python3.10-pyarrow>                 )
python3.10-pyarrow> >           subprocess.check_call([sys.executable, '-c', code],
python3.10-pyarrow>                                   stdout=subprocess.PIPE,
python3.10-pyarrow>                                   env=subprocess_env)
python3.10-pyarrow> pyarrow/tests/test_cython.py:157:
python3.10-pyarrow> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
python3.10-pyarrow> popenargs = (['/nix/store/fbll01zhyfffmshwrrzphmbscmnimn3f-python3-3.10.9/bin/python3.10', '-c', 'if 1:\n            import sys\n ..._null_array(5)\n            assert mod.get_array_length(arr) == 5\n            assert arr.null_count == 5\n        '],)
python3.10-pyarrow> kwargs = {'env': {'AR': 'ar', 'ARROW_HOME': '/nix/store/gvzv197a7rgx06f3hk1mb3pfnz6xkp4v-arrow-cpp-10.0.1', 'ARROW_TEST_DATA': '/nix/store/rwc1jakxaqyp6i9hf1p7bi2i6an5ph3b-arrow-testing/data', 'AS': 'as', ...}, 'stdout': -1}
python3.10-pyarrow> retcode = 1
python3.10-pyarrow> cmd = ['/nix/store/fbll01zhyfffmshwrrzphmbscmnimn3f-python3-3.10.9/bin/python3.10', '-c', 'if 1:\n            import sys\n  ...ke_null_array(5)\n            assert mod.get_array_length(arr) == 5\n            assert arr.null_count == 5\n        ']
python3.10-pyarrow>     def check_call(*popenargs, **kwargs):
python3.10-pyarrow>         """Run command with arguments.  Wait for command to complete.  If
python3.10-pyarrow>         the exit code was zero then return, otherwise raise
python3.10-pyarrow>         CalledProcessError.  The CalledProcessError object will have the
python3.10-pyarrow>         return code in the returncode attribute.
python3.10-pyarrow>         The arguments are the same as for the call function.  Example:
python3.10-pyarrow>         check_call(["ls", "-l"])
python3.10-pyarrow>         """
python3.10-pyarrow>         retcode = call(*popenargs, **kwargs)
python3.10-pyarrow>         if retcode:
python3.10-pyarrow>             cmd = kwargs.get("args")
python3.10-pyarrow>             if cmd is None:
python3.10-pyarrow>                 cmd = popenargs[0]
python3.10-pyarrow> >           raise CalledProcessError(retcode, cmd)
python3.10-pyarrow> E           subprocess.CalledProcessError: Command '['/nix/store/fbll01zhyfffmshwrrzphmbscmnimn3f-python3-3.10.9/bin/python3.10', '-c', "if 1:\n            import sys\n            import os\n\n            try:\n                # Add dll directory was added on python 3.8\n                # and is required in order to find extra DLLs\n                # only for win32\n                for dir in ['/nix/store/flyjhifsp7ql63rd15dgv8dy60ga5slb-python3.10-pyarrow-10.0.1/lib/python3.10/site-packages/pyarrow', '/nix/store/gvzv197a7rgx06f3hk1mb3pfnz6xkp4v-arrow-cpp-10.0.1/lib']:\n                    os.add_dll_directory(dir)\n            except AttributeError:\n                pass\n\n            mod = __import__('pyarrow_cython_example')\n            arr = mod.make_null_array(5)\n            assert mod.get_array_length(arr) == 5\n            assert arr.null_count == 5\n        "]' returned non-zero exit status 1.
python3.10-pyarrow> /nix/store/fbll01zhyfffmshwrrzphmbscmnimn3f-python3-3.10.9/lib/python3.10/subprocess.py:369: CalledProcessError
python3.10-pyarrow> ----------------------------- Captured stdout call -----------------------------
python3.10-pyarrow> Compiling pyarrow_cython_example.pyx because it changed.
python3.10-pyarrow> [1/1] Cythonizing pyarrow_cython_example.pyx
python3.10-pyarrow> Extension module: <setuptools.extension.Extension('pyarrow_cython_example') at 0x10effc280> ['/nix/store/7970xzyh5awa1vnzr5mbv9aind9hhkd4-python3.10-numpy-1.23.3/lib/python3.10/site-packages/numpy/core/include', '/nix/store/flyjhifsp7ql63rd15dgv8dy60ga5slb-python3.10-pyarrow-10.0.1/lib/python3.10/site-packages/pyarrow/include'] ['arrow', 'arrow_python'] ['/nix/store/flyjhifsp7ql63rd15dgv8dy60ga5slb-python3.10-pyarrow-10.0.1/lib/python3.10/site-packages/pyarrow', '/nix/store/gvzv197a7rgx06f3hk1mb3pfnz6xkp4v-arrow-cpp-10.0.1/lib']
python3.10-pyarrow> running build_ext
python3.10-pyarrow> building 'pyarrow_cython_example' extension
python3.10-pyarrow> creating build
python3.10-pyarrow> creating build/temp.macosx-10.12-x86_64-cpython-310
python3.10-pyarrow> clang++ -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/nix/store/x3305j9r8kfvgz19vh6k1k28maa1sday-libxcrypt-4.4.33/include -I/nix/store/7970xzyh5awa1vnzr5mbv9aind9hhkd4-python3.10-numpy-1.23.3/lib/python3.10/site-packages/numpy/core/include -I/nix/store/flyjhifsp7ql63rd15dgv8dy60ga5slb-python3.10-pyarrow-10.0.1/lib/python3.10/site-packages/pyarrow/include -I/nix/store/fbll01zhyfffmshwrrzphmbscmnimn3f-python3-3.10.9/include/python3.10 -c pyarrow_cython_example.cpp -o build/temp.macosx-10.12-x86_64-cpython-310/pyarrow_cython_example.o -std=c++17
python3.10-pyarrow> creating build/lib.macosx-10.12-x86_64-cpython-310
python3.10-pyarrow> clang++ -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/nix/store/x3305j9r8kfvgz19vh6k1k28maa1sday-libxcrypt-4.4.33/include -bundle -undefined dynamic_lookup build/temp.macosx-10.12-x86_64-cpython-310/pyarrow_cython_example.o -L/nix/store/flyjhifsp7ql63rd15dgv8dy60ga5slb-python3.10-pyarrow-10.0.1/lib/python3.10/site-packages/pyarrow -L/nix/store/gvzv197a7rgx06f3hk1mb3pfnz6xkp4v-arrow-cpp-10.0.1/lib -L/nix/store/fbll01zhyfffmshwrrzphmbscmnimn3f-python3-3.10.9/lib -larrow -larrow_python -o build/lib.macosx-10.12-x86_64-cpython-310/pyarrow_cython_example.cpython-310-darwin.so
python3.10-pyarrow> copying build/lib.macosx-10.12-x86_64-cpython-310/pyarrow_cython_example.cpython-310-darwin.so ->
python3.10-pyarrow> ----------------------------- Captured stderr call -----------------------------
python3.10-pyarrow> clang-11: warning: argument unused during compilation: '-fno-strict-overflow' [-Wunused-command-line-argument]
python3.10-pyarrow> pyarrow_cython_example.cpp:3756:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_buffer' [-Wunused-variable]
python3.10-pyarrow> static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_buffer)(std::shared_ptr< arrow::Buffer>  const &); /*proto*/
python3.10-pyarrow>                    ^
python3.10-pyarrow> pyarrow_cython_example.cpp:3757:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_resizable_buffer' [-Wunused-variable]
python3.10-pyarrow> static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_resizable_buffer)(std::shared_ptr< arrow::ResizableBuffer>  const &); /*proto*/
python3.10-pyarrow>                    ^
python3.10-pyarrow> pyarrow_cython_example.cpp:3758:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_data_type' [-Wunused-variable]
python3.10-pyarrow> static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_data_type)(std::shared_ptr< arrow::DataType>  const &); /*proto*/
python3.10-pyarrow>                    ^
python3.10-pyarrow> pyarrow_cython_example.cpp:3759:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_field' [-Wunused-variable]
python3.10-pyarrow> static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_field)(std::shared_ptr< arrow::Field>  const &); /*proto*/
python3.10-pyarrow>                    ^
python3.10-pyarrow> pyarrow_cython_example.cpp:3760:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_schema' [-Wunused-variable]
python3.10-pyarrow> static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_schema)(std::shared_ptr< arrow::Schema>  const &); /*proto*/
python3.10-pyarrow>                    ^
python3.10-pyarrow> pyarrow_cython_example.cpp:3763:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_chunked_array' [-Wunused-variable]
python3.10-pyarrow> static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_chunked_array)(std::shared_ptr< arrow::ChunkedArray>  const &); /*proto*/
python3.10-pyarrow>                    ^
python3.10-pyarrow> pyarrow_cython_example.cpp:3764:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_sparse_coo_tensor' [-Wunused-variable]
python3.10-pyarrow> static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_sparse_coo_tensor)(std::shared_ptr< arrow::SparseCOOTensor>  const &); /*proto*/
python3.10-pyarrow>                    ^
python3.10-pyarrow> pyarrow_cython_example.cpp:3765:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_sparse_csc_matrix' [-Wunused-variable]
python3.10-pyarrow> static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_sparse_csc_matrix)(std::shared_ptr< arrow::SparseCSCMatrix>  const &); /*proto*/
python3.10-pyarrow>                    ^
python3.10-pyarrow> pyarrow_cython_example.cpp:3766:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_sparse_csf_tensor' [-Wunused-variable]
python3.10-pyarrow> static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_sparse_csf_tensor)(std::shared_ptr< arrow::SparseCSFTensor>  const &); /*proto*/
python3.10-pyarrow>                    ^
python3.10-pyarrow> pyarrow_cython_example.cpp:3767:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_sparse_csr_matrix' [-Wunused-variable]
python3.10-pyarrow> static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_sparse_csr_matrix)(std::shared_ptr< arrow::SparseCSRMatrix>  const &); /*proto*/
python3.10-pyarrow>                    ^
python3.10-pyarrow> pyarrow_cython_example.cpp:3768:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_tensor' [-Wunused-variable]
python3.10-pyarrow> static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_tensor)(std::shared_ptr< arrow::Tensor>  const &); /*proto*/
python3.10-pyarrow>                    ^
python3.10-pyarrow> pyarrow_cython_example.cpp:3769:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_batch' [-Wunused-variable]
python3.10-pyarrow> static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_batch)(std::shared_ptr< arrow::RecordBatch>  const &); /*proto*/
python3.10-pyarrow>                    ^
python3.10-pyarrow> pyarrow_cython_example.cpp:3770:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_table' [-Wunused-variable]
python3.10-pyarrow> static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_table)(std::shared_ptr< arrow::Table>  const &); /*proto*/
python3.10-pyarrow>                    ^
python3.10-pyarrow> pyarrow_cython_example.cpp:3771:43: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_buffer' [-Wunused-variable]
python3.10-pyarrow> static std::shared_ptr< arrow::Buffer>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_buffer)(PyObject *); /*proto*/
python3.10-pyarrow>                                           ^
python3.10-pyarrow> pyarrow_cython_example.cpp:3773:42: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_field' [-Wunused-variable]
python3.10-pyarrow> static std::shared_ptr< arrow::Field>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_field)(PyObject *); /*proto*/
python3.10-pyarrow>                                          ^
python3.10-pyarrow> pyarrow_cython_example.cpp:3774:43: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_schema' [-Wunused-variable]
python3.10-pyarrow> static std::shared_ptr< arrow::Schema>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_schema)(PyObject *); /*proto*/
python3.10-pyarrow>                                           ^
python3.10-pyarrow> pyarrow_cython_example.cpp:3777:49: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_chunked_array' [-Wunused-variable]
python3.10-pyarrow> static std::shared_ptr< arrow::ChunkedArray>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_chunked_array)(PyObject *); /*proto*/
python3.10-pyarrow>                                                 ^
python3.10-pyarrow> pyarrow_cython_example.cpp:3778:52: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_sparse_coo_tensor' [-Wunused-variable]
python3.10-pyarrow> static std::shared_ptr< arrow::SparseCOOTensor>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_sparse_coo_tensor)(PyObject *); /*proto*/
python3.10-pyarrow>                                                    ^
python3.10-pyarrow> pyarrow_cython_example.cpp:3779:52: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_sparse_csc_matrix' [-Wunused-variable]
python3.10-pyarrow> static std::shared_ptr< arrow::SparseCSCMatrix>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_sparse_csc_matrix)(PyObject *); /*proto*/
python3.10-pyarrow>                                                    ^
python3.10-pyarrow> pyarrow_cython_example.cpp:3780:52: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_sparse_csf_tensor' [-Wunused-variable]
python3.10-pyarrow> static std::shared_ptr< arrow::SparseCSFTensor>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_sparse_csf_tensor)(PyObject *); /*proto*/
python3.10-pyarrow>                                                    ^
python3.10-pyarrow> pyarrow_cython_example.cpp:3781:52: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_sparse_csr_matrix' [-Wunused-variable]
python3.10-pyarrow> static std::shared_ptr< arrow::SparseCSRMatrix>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_sparse_csr_matrix)(PyObject *); /*proto*/
python3.10-pyarrow>                                                    ^
python3.10-pyarrow> pyarrow_cython_example.cpp:3782:43: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_tensor' [-Wunused-variable]
python3.10-pyarrow> static std::shared_ptr< arrow::Tensor>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_tensor)(PyObject *); /*proto*/
python3.10-pyarrow>                                           ^
python3.10-pyarrow> pyarrow_cython_example.cpp:3783:48: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_batch' [-Wunused-variable]
python3.10-pyarrow> static std::shared_ptr< arrow::RecordBatch>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_batch)(PyObject *); /*proto*/
python3.10-pyarrow>                                                ^
python3.10-pyarrow> pyarrow_cython_example.cpp:3784:42: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_table' [-Wunused-variable]
python3.10-pyarrow> static std::shared_ptr< arrow::Table>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_table)(PyObject *); /*proto*/
python3.10-pyarrow>                                          ^
python3.10-pyarrow> 24 warnings generated.
python3.10-pyarrow> Traceback (most recent call last):
python3.10-pyarrow>   File "<string>", line 14, in <module>
python3.10-pyarrow> ImportError: dlopen(/private/tmp/nix-build-python3.10-pyarrow-10.0.1.drv-1/pytest-of-_nixbld1/pytest-0/test_cython_api0/pyarrow_cython_example.cpython-310-darwin.so, 0x0002): Library not loaded: '@rpath/libarrow_python.1000.dylib'
python3.10-pyarrow>   Referenced from: '/private/tmp/nix-build-python3.10-pyarrow-10.0.1.drv-1/pytest-of-_nixbld1/pytest-0/test_cython_api0/pyarrow_cython_example.cpython-310-darwin.so'
python3.10-pyarrow>   Reason: tried: '/nix/store/446dayrafwsc1vcqpgxc4l5wh2cljgbb-swift-corefoundation-unstable-2018-09-14/Library/Frameworks/libarrow_python.1000.dylib' (no such file), '/nix/store/446dayrafwsc1vcqpgxc4l5wh2cljgbb-swift-corefoundation-unstable-2018-09-14/Library/Frameworks/libarrow_python.1000.dylib' (no such file), '/nix/store/446dayrafwsc1vcqpgxc4l5wh2cljgbb-swift-corefoundation-unstable-2018-09-14/Library/Frameworks/libarrow_python.1000.dylib' (no such file), '/nix/store/446dayrafwsc1vcqpgxc4l5wh2cljgbb-swift-corefoundation-unstable-2018-09-14/Library/Frameworks/libarrow_python.1000.dylib' (no such file), '/usr/local/lib/libarrow_python.1000.dylib' (no such file), '/usr/lib/libarrow_python.1000.dylib' (no such file)

Disabling the test with "--deselect=pyarrow/tests/test_cython.py::test_cython_api" fixes the build.

@tobim
Copy link
Contributor

tobim commented Jan 15, 2023

Would be really nice to get this merged.

@cpcloud
Copy link
Contributor Author

cpcloud commented Jan 15, 2023

I don't have access to a mac to debug the problem, how should we go about getting it merged?

@tobim
Copy link
Contributor

tobim commented Jan 15, 2023

It should be fine to disable the test as I wrote at the end of my comment.

@cpcloud
Copy link
Contributor Author

cpcloud commented Jan 16, 2023

Ah, sorry I missed that! Thanks.

@ofborg ofborg bot requested a review from veprbl January 21, 2023 10:07
@tobim
Copy link
Contributor

tobim commented Jan 28, 2023 •
edited
Loading

@veprbl I think this is ready to merge. Time to work on Arrow 11 😉 .

@veprbl
Copy link
Member

veprbl commented Jan 28, 2023

@veprbl I think this is ready to merge. Time to work on Arrow 11 😉 .

Ahaha, yes. Unfortunately the merge conflict popped up right after previous push.

@tobim tobim mentioned this pull request Jan 29, 2023
Merged
13 tasks
@cpcloud cpcloud closed this Feb 9, 2023
@cpcloud cpcloud deleted the arrow-10 branch February 9, 2023 14:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@FRidh FRidh Awaiting requested review from FRidh

@jonringer jonringer Awaiting requested review from jonringer

@LnL7 LnL7 Awaiting requested review from LnL7

@tobim tobim Awaiting requested review from tobim

@marsam marsam Awaiting requested review from marsam

@veprbl veprbl Awaiting requested review from veprbl

Assignees

No one assigned

Labels

6.topic: python Python is a high-level, general-purpose programming language. 10.rebuild-darwin: 101-500 This PR causes between 101 and 500 packages to rebuild on Darwin. 10.rebuild-linux: 101-500 This PR causes between 101 and 500 packages to rebuild on Linux. 11.by: package-maintainer This PR was created by a maintainer of all the package it changes.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@cpcloud @tobim @veprbl @marsam