Hi, team.
In the process of working on #7728, I noticed python/python/benchmarks/test_random_access.py does not work for me, and it is not a pytest file.
lance/python on ξ main π v3.13.13 β― uv run pytest python/benchmarks/ -m "not slow"
====================================================================================================== test session starts =======================================================================================================
platform linux -- Python 3.13.13, pytest-8.4.2, pluggy-1.6.0
benchmark: 5.1.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/moco/repo/mocobeta/lance/python
configfile: pyproject.toml
plugins: benchmark-5.1.0
collected 295 items / 1 error / 2 deselected / 293 selected
============================================================================================================= ERRORS =============================================================================================================
____________________________________________________________________________________ ERROR collecting python/benchmarks/test_random_access.py ____________________________________________________________________________________
python/benchmarks/test_random_access.py:12: in <module>
tab = pq.read_table("~/lineitemsf1.snappy.parquet")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/pyarrow/parquet/core.py:1891: in read_table
dataset = ParquetDataset(
.venv/lib/python3.13/site-packages/pyarrow/parquet/core.py:1471: in __init__
self._dataset = ds.dataset(path_or_paths, filesystem=filesystem,
.venv/lib/python3.13/site-packages/pyarrow/dataset.py:790: in dataset
return _filesystem_dataset(source, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/pyarrow/dataset.py:472: in _filesystem_dataset
fs, paths_or_selector = _ensure_single_source(source, filesystem)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/pyarrow/dataset.py:437: in _ensure_single_source
raise FileNotFoundError(path)
E FileNotFoundError: /home/moco/lineitemsf1.snappy.parquet
==================================================================================================== short test summary info =====================================================================================================
ERROR python/benchmarks/test_random_access.py - FileNotFoundError: /home/moco/lineitemsf1.snappy.parquet
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================================================================= 2 deselected, 1 error in 1.49s =================================================================================================
This prevents running the whole benchmark under python/python/benchmark. I know that one can use pytest's --ignore option to exclude the file, but can we probably move it elsewhere or archive the possibly invalid test file? By excluding this file, I can run the whole benchmark.
lance/python on ξ main π v3.13.13 β― uv run pytest python/benchmarks/ -m "not slow" --ignore python/benchmarks/test_random_access.py
...
==================================================================================================== short test summary info =====================================================================================================
FAILED python/benchmarks/test_file.py::test_sample_integer[2_0] - assert 10 == 10000000
FAILED python/benchmarks/test_file.py::test_sample_integer[2_1] - assert 10 == 10000000
FAILED python/benchmarks/test_file.py::test_sample_integer[2_2] - assert 10 == 10000000
FAILED python/benchmarks/test_file.py::test_sample_nested_integer - assert 10 == 10000000
FAILED python/benchmarks/test_index.py::test_create_ivf_pq_torch_cpu - ModuleNotFoundError: torch.device requires 'torch' module to be installed
FAILED python/benchmarks/test_index.py::test_create_ivf_pq_torch_cpu_one_pass - ModuleNotFoundError: torch.device requires 'torch' module to be installed
FAILED python/benchmarks/test_index.py::test_optimize_index[12000-5-256] - ValueError: num_partitions and num_sub_vectors are required for IVF_PQ
FAILED python/benchmarks/test_index.py::test_optimize_index[12000-5-512] - ValueError: num_partitions and num_sub_vectors are required for IVF_PQ
ERROR python/benchmarks/test_index.py::test_transform_vectors_no_precomputed_parts
ERROR python/benchmarks/test_search.py::test_geo_rtree[no_index] - ModuleNotFoundError: No module named 'geoarrow'
ERROR python/benchmarks/test_search.py::test_geo_rtree[with_index] - ModuleNotFoundError: No module named 'geoarrow'
==================================================================== 8 failed, 280 passed, 2 skipped, 2 deselected, 6 warnings, 3 errors in 431.23s (0:07:11) ====================================================================
Does anyone have any ideas/advice on how to handle this file?
Hi, team.
In the process of working on #7728, I noticed
python/python/benchmarks/test_random_access.pydoes not work for me, and it is not apytestfile.This prevents running the whole benchmark under
python/python/benchmark. I know that one can use pytest's--ignoreoption to exclude the file, but can we probably move it elsewhere or archive the possibly invalid test file? By excluding this file, I can run the whole benchmark.Does anyone have any ideas/advice on how to handle this file?