Skip to content

test_reproject_error_propagation fails on Python 3.14 #3357

@QuLogic

Description

@QuLogic

Expected behavior and actual behavior.

Python 3.14b1 has changed the default multiprocessing start method from fork to forkserver: https://docs.python.org/dev/whatsnew/3.14.html#multiprocessing

This appears to break the test setup which can't be pickled:

______________ ERROR at setup of test_reproject_error_propagation ______________

data = local('/tmp/pytest-of-mockbuild/pytest-0/test_reproject_error_propagati0')

    @pytest.mark.network
    @pytest.fixture
    def http_error_server(data):
        """Serves files from the test data directory, poorly."""
        import functools
        import multiprocessing
        import http.server
    
        Handler = functools.partial(RangeRequestErrorHandler, directory=str(data))
        httpd = http.server.HTTPServer(("", 0), Handler)
        p = multiprocessing.Process(target=httpd.serve_forever)
>       p.start()

tests/test_warp.py:2126: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib64/python3.14/multiprocessing/process.py:121: in start
    self._popen = self._Popen(self)
/usr/lib64/python3.14/multiprocessing/context.py:224: in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
/usr/lib64/python3.14/multiprocessing/context.py:300: in _Popen
    return Popen(process_obj)
/usr/lib64/python3.14/multiprocessing/popen_forkserver.py:35: in __init__
    super().__init__(process_obj)
/usr/lib64/python3.14/multiprocessing/popen_fork.py:20: in __init__
    self._launch(process_obj)
/usr/lib64/python3.14/multiprocessing/popen_forkserver.py:47: in _launch
    reduction.dump(process_obj, buf)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

obj = <Process name='Process-9' parent=2132 initial>
file = <_io.BytesIO object at 0x7f384f02fa60>, protocol = None

    def dump(obj, file, protocol=None):
        '''Replacement for pickle.dump() using ForkingPickler.'''
>       ForkingPickler(file, protocol).dump(obj)
E       TypeError: cannot pickle '_thread.lock' object
E       when serializing dict item '_lock'
E       when serializing threading.Condition state
E       when serializing threading.Condition object
E       when serializing dict item '_cond'
E       when serializing threading.Event state
E       when serializing threading.Event object
E       when serializing dict item '_BaseServer__is_shut_down'
E       when serializing http.server.HTTPServer state
E       when serializing http.server.HTTPServer object
E       when serializing tuple item 0
E       when serializing method reconstructor arguments
E       when serializing method object
E       when serializing dict item '_target'
E       when serializing multiprocessing.context.Process state
E       when serializing multiprocessing.context.Process object

I guess maybe the start method should explicitly be set to fork?

Environment Information

  • rasterio version (python -c "import rasterio; print(rasterio.__version__)") 1.4.3
  • GDAL version (python -c "import rasterio; print(rasterio.__gdal_version__)") n/a
  • Python version (python -c "import sys; print(sys.version.replace('\n', ' '))") 3.14.0b1
  • Operation System Information (python -c "import platform; print(platform.platform())") Fedora Rawhide

Installation Method

From sdist.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions