Repository navigation
AttributeError: Can't pickle local object 'lazy_call.<locals>._handler' #22
Description
Hey there,
Running inside Github actions, using Poetry, I get this error:
packages/unittest_parallel/main.py", line 114, in main
results = pool.map(test_manager.run_tests, test_suites)
File "/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/multiprocessing/pool.py", line 367, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/multiprocessing/pool.py", line 774, in get
raise self._value
File "/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/multiprocessing/pool.py", line 540, in _handle_tasks
put(task)
File "/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/multiprocessing/connection.py", line 206, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/multiprocessing/reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
AttributeError: Can't pickle local object 'lazy_call.<locals>._handler'
I quick search came up with this which suggests changing the process forking type from spawn to fork.
I tried changing multiprocessing_context = multiprocessing.get_context(method='spawn') to fork but it didn't help.
I should mention that this is running on a runner using arm64, not x86_64...
Any thoughts on this?
The command line was very simple:
poetry run unittest-parallel -s <some_dir_with_tests>
thanks!
Activity
amirse commentedon Oct 8, 2024
Hmm, OK nevermind about the arm64, since I just tried running the test on my machine and it's an x86_64 Ubuntu 24.04. Same deal..
craigahobbs commentedon Oct 8, 2024
A couple of thoughts:
unittest-parallel moved from fork to spawn in version 1.5 (see Some frameworks deadlock when executed with unittest-parallel #11 and Use spawn to start processes and add an option to use new processes. #12). Can you use unittest-parallel 1.4 to see if that fixes the issue?
issue Occasional exception in multiprocessing when running in Python 3.7 #15 reported an occasional pickling issue with only Python 3.7. After some investigation, I theorized this was a pickle issue fixed in Python 3.8. Perhaps it wasn't.
Does the error happen every test run, or is it occasional, like #15? Does the issue happen in Python 3.11 or 3.12?