Skip to content

BUG: DataSource methods do not consistently accept pathlib.Path inputs #31909

Description

@VectorPeak

Describe the issue:

In #23822, several numpy.lib._datasource docstrings were updated to document path parameters as accepting str or pathlib.Path. However, some of those documented pathlib.Path inputs are not handled consistently at runtime because the implementation can still pass the original object into string-specific path logic before normalizing it.

The affected cases are local pathlib.Path inputs for datasource entry points that otherwise accept string local paths:

  • DataSource.abspath(Path(...))
  • DataSource.exists(Path(...))
  • DataSource.open(Path(...))
  • the module-level numpy.lib._datasource.open(Path(...))
  • Repository path resolution, for example Repository(...).open(Path(...)), Repository(...).exists(Path(...)), and Repository(...).abspath(Path(...))

The failure mode is not that URL handling or caching needs new behavior. The problem is narrower: a documented local path input can reach string-only operations such as path.split(...) before being converted to a filesystem path string.

A proposed fix is available in #31906. It normalizes path-like inputs with os.fspath() before the existing URL, cache, absolute-path, and file-opening logic runs.

Reproduce the code example:

from pathlib import Path
import tempfile

from numpy.lib import _datasource as datasource

with tempfile.TemporaryDirectory() as tmpdir:
    path = Path(tmpdir) / "data.txt"
    path.write_text("ok", encoding="utf-8")

    ds = datasource.DataSource(tmpdir)

    # These should behave the same as passing str(path), as documented.
    print(ds.abspath(path))
    print(ds.exists(path))

    with ds.open(path) as f:
        print(f.read())

    repo = datasource.Repository(Path(tmpdir), tmpdir)

    # Repository path resolution also reaches string-specific logic.
    print(repo.abspath(Path("data.txt")))
    print(repo.exists(Path("data.txt")))

    with repo.open(Path("data.txt")) as f:
        print(f.read())

Error message:

For affected code paths, a pathlib.Path object can be used where the implementation expects a string and calls string-only methods such as .split(...).

AttributeError: 'PosixPath' object has no attribute 'split'

Python and NumPy Versions:

Observed while testing the current development branch with Python 3.12.13.

NumPy: development branch around #31906
Python: 3.12.13

Runtime Environment:

Not included. This is a small path-normalization issue in numpy.lib._datasource; it does not appear to depend on BLAS, SIMD, or platform-specific NumPy runtime configuration.

How does this issue affect you or how did you find it:

This is an API/documentation consistency issue. Since #23822 documented these datasource paths as accepting pathlib.Path, users can reasonably expect Path inputs to work in the same places as string local paths. In practice, some of the documented cases were not covered by tests and still assume plain string operations internally.

The intended behavior is to normalize local path-like inputs at the datasource boundary while preserving the existing behavior for URL strings, cache lookup order, write-mode URL rejection, compressed-file opening, and sandboxing.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions