TST: Fix test fixture for test_reproject_error_propagation#3360
TST: Fix test fixture for test_reproject_error_propagation#3360snowman2 merged 1 commit intorasterio:mainfrom
Conversation
|
@QuLogic can you say more about what places this doesn't work and why pickling is important? |
|
Pickling is something that
For this, I'm not entirely sure the cause yet, but the |
QuLogic
force-pushed
the
fix-test-fixture
branch
from
2bcd896 to
97f48db
Compare
Pickling the HTTP server requires the 'fork' multiprocessing context, and in some places, `HTTPServer.server_name` returns something that doesn't work with curl. So force the 'fork' mode, and use `HTTPServer.server_address` instead. Also, remove the `network` marker, as it does nothing for fixtures and isn't required since that marker signifies _external_ networking.
QuLogic
force-pushed
the
fix-test-fixture
branch
from
97f48db to
b80bb64
Compare
To add on here, TorchGeo uses rasterio in most of its data loaders to load data in parallel. This requires multiprocessing, which requires in-memory objects to be pickle-able in many multiprocessing contexts. I don't think we specifically rely on HTTP server, so as long as the other components work with the new multiprocessing context then we're fine. |
|
Thanks @QuLogic 👍 |
Pickling the HTTP server requires the 'fork' multiprocessing context, and in some places, `HTTPServer.server_name` returns something that doesn't work with curl. So force the 'fork' mode, and use `HTTPServer.server_address` instead. Also, remove the `network` marker, as it does nothing for fixtures and isn't required since that marker signifies _external_ networking.
4 participants
Pickling the HTTP server requires the 'fork' multiprocessing context, and in some places,
HTTPServer.server_namereturns something that doesn't work with curl. So force the 'fork' mode, and useHTTPServer.server_addressinstead.Also, remove the
networkmarker, as it does nothing for fixtures and isn't required since that marker signifies external networking.Fixes #3357
Fixes #3403