MacOS Mojave, Python 3.8: "AttributeError: Can't pickle local object 'fork.<locals>.child_process'" #14102
Description
I tried to install a package on MacOS Mojave using Python 3.8 installed from MacPorts. I got an error related to "pickle"ing an object.
The issue does not reproduce with Python 3.7. It reproduces on the latest commit of develop.
Steps to reproduce the issue
$ sudo port select --set python3 python38
$ spack install <package>Error Message
$ spack -d install googletest%clang
==> [2019-12-10-16:40:53.367136] Reading config file /Users/agaspar/Code/spack/etc/spack/defaults/modules.yaml
==> [2019-12-10-16:40:53.375856] Reading config file /Users/agaspar/Code/spack/etc/spack/defaults/darwin/modules.yaml
==> [2019-12-10-16:40:53.378937] Reading config file /Users/agaspar/.spack/darwin/modules.yaml
==> [2019-12-10-16:40:53.382033] Reading config file /Users/agaspar/Code/spack/etc/spack/defaults/config.yaml
==> [2019-12-10-16:40:53.402445] Imported install from built-in commands
==> [2019-12-10-16:40:53.403393] Imported install from built-in commands
==> [2019-12-10-16:40:53.423052] Reading config file /Users/agaspar/Code/spack/etc/spack/defaults/repos.yaml
==> [2019-12-10-16:40:53.426926] Reading config file /Users/agaspar/.spack/darwin/repos.yaml
==> [2019-12-10-16:40:53.590667] Reading config file /Users/agaspar/Code/spack/etc/spack/defaults/packages.yaml
==> [2019-12-10-16:40:53.602863] Reading config file /Users/agaspar/Code/spack/etc/spack/defaults/darwin/packages.yaml
==> [2019-12-10-16:40:53.609408] Reading config file /Users/agaspar/.spack/packages.yaml
==> [2019-12-10-16:40:53.632981] WRITE LOCK: /Users/agaspar/.spack/cache/providers/.builtin-index.json.lock[0:0] [Acquiring]
==> [2019-12-10-16:40:53.633665] WRITE LOCK: /Users/agaspar/.spack/cache/providers/.builtin-index.json.lock[0:0] [Acquired]
==> [2019-12-10-16:40:59.187669] WRITE LOCK: /Users/agaspar/.spack/cache/providers/.builtin-index.json.lock[0:0] [Released]
==> [2019-12-10-16:40:59.191428] WRITE LOCK: /Users/agaspar/.spack/cache/tags/.builtin-index.json.lock[0:0] [Acquiring]
==> [2019-12-10-16:40:59.192032] WRITE LOCK: /Users/agaspar/.spack/cache/tags/.builtin-index.json.lock[0:0] [Acquired]
==> [2019-12-10-16:40:59.320890] WRITE LOCK: /Users/agaspar/.spack/cache/tags/.builtin-index.json.lock[0:0] [Released]
==> [2019-12-10-16:40:59.325895] WRITE LOCK: /Users/agaspar/.spack/cache/patches/.builtin-index.json.lock[0:0] [Acquiring]
==> [2019-12-10-16:40:59.326356] WRITE LOCK: /Users/agaspar/.spack/cache/patches/.builtin-index.json.lock[0:0] [Acquired]
==> [2019-12-10-16:40:59.913941] WRITE LOCK: /Users/agaspar/.spack/cache/patches/.builtin-index.json.lock[0:0] [Released]
==> [2019-12-10-16:40:59.920255] READ LOCK: /Users/agaspar/.spack/cache/providers/.eap-cpp-repo-index.json.lock[0:0] [Acquiring]
==> [2019-12-10-16:40:59.920748] READ LOCK: /Users/agaspar/.spack/cache/providers/.eap-cpp-repo-index.json.lock[0:0] [Acquired]
==> [2019-12-10-16:40:59.921247] READ LOCK: /Users/agaspar/.spack/cache/providers/.eap-cpp-repo-index.json.lock[0:0] [Released]
==> [2019-12-10-16:40:59.921705] READ LOCK: /Users/agaspar/.spack/cache/tags/.eap-cpp-repo-index.json.lock[0:0] [Acquiring]
==> [2019-12-10-16:40:59.921978] READ LOCK: /Users/agaspar/.spack/cache/tags/.eap-cpp-repo-index.json.lock[0:0] [Acquired]
==> [2019-12-10-16:40:59.922417] READ LOCK: /Users/agaspar/.spack/cache/tags/.eap-cpp-repo-index.json.lock[0:0] [Released]
==> [2019-12-10-16:40:59.922960] READ LOCK: /Users/agaspar/.spack/cache/patches/.eap-cpp-repo-index.json.lock[0:0] [Acquiring]
==> [2019-12-10-16:40:59.923237] READ LOCK: /Users/agaspar/.spack/cache/patches/.eap-cpp-repo-index.json.lock[0:0] [Acquired]
==> [2019-12-10-16:40:59.931963] READ LOCK: /Users/agaspar/.spack/cache/patches/.eap-cpp-repo-index.json.lock[0:0] [Released]
==> [2019-12-10-16:40:59.933605] READ LOCK: /Users/agaspar/.spack/cache/providers/.agaspar-spack-repo-index.json.lock[0:0] [Acquiring]
==> [2019-12-10-16:40:59.934507] READ LOCK: /Users/agaspar/.spack/cache/providers/.agaspar-spack-repo-index.json.lock[0:0] [Acquired]
==> [2019-12-10-16:40:59.935006] READ LOCK: /Users/agaspar/.spack/cache/providers/.agaspar-spack-repo-index.json.lock[0:0] [Released]
==> [2019-12-10-16:40:59.935556] READ LOCK: /Users/agaspar/.spack/cache/tags/.agaspar-spack-repo-index.json.lock[0:0] [Acquiring]
==> [2019-12-10-16:40:59.935896] READ LOCK: /Users/agaspar/.spack/cache/tags/.agaspar-spack-repo-index.json.lock[0:0] [Acquired]
==> [2019-12-10-16:40:59.936364] READ LOCK: /Users/agaspar/.spack/cache/tags/.agaspar-spack-repo-index.json.lock[0:0] [Released]
==> [2019-12-10-16:40:59.936862] READ LOCK: /Users/agaspar/.spack/cache/patches/.agaspar-spack-repo-index.json.lock[0:0] [Acquiring]
==> [2019-12-10-16:40:59.937520] READ LOCK: /Users/agaspar/.spack/cache/patches/.agaspar-spack-repo-index.json.lock[0:0] [Acquired]
==> [2019-12-10-16:40:59.937955] READ LOCK: /Users/agaspar/.spack/cache/patches/.agaspar-spack-repo-index.json.lock[0:0] [Released]
==> [2019-12-10-16:40:59.938516] READ LOCK: /Users/agaspar/.spack/cache/providers/.sds-index.json.lock[0:0] [Acquiring]
==> [2019-12-10-16:40:59.938796] READ LOCK: /Users/agaspar/.spack/cache/providers/.sds-index.json.lock[0:0] [Acquired]
==> [2019-12-10-16:40:59.939228] READ LOCK: /Users/agaspar/.spack/cache/providers/.sds-index.json.lock[0:0] [Released]
==> [2019-12-10-16:40:59.939772] READ LOCK: /Users/agaspar/.spack/cache/tags/.sds-index.json.lock[0:0] [Acquiring]
==> [2019-12-10-16:40:59.940071] READ LOCK: /Users/agaspar/.spack/cache/tags/.sds-index.json.lock[0:0] [Acquired]
==> [2019-12-10-16:40:59.940524] READ LOCK: /Users/agaspar/.spack/cache/tags/.sds-index.json.lock[0:0] [Released]
==> [2019-12-10-16:40:59.941433] READ LOCK: /Users/agaspar/.spack/cache/patches/.sds-index.json.lock[0:0] [Acquiring]
==> [2019-12-10-16:40:59.941899] READ LOCK: /Users/agaspar/.spack/cache/patches/.sds-index.json.lock[0:0] [Acquired]
==> [2019-12-10-16:40:59.942576] READ LOCK: /Users/agaspar/.spack/cache/patches/.sds-index.json.lock[0:0] [Released]
==> [2019-12-10-16:40:59.947954] Reading config file /Users/agaspar/.spack/darwin/compilers.yaml
==> [2019-12-10-16:41:00.015384] Warning: [email protected] cannot build optimized binaries for "skylake". Using best target possible: "x86_64"
==> [2019-12-10-16:41:00.066119] DATABASE LOCK TIMEOUT: 120s
==> [2019-12-10-16:41:00.066246] PACKAGE LOCK TIMEOUT: No timeout
==> [2019-12-10-16:41:00.069298] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquiring]
==> [2019-12-10-16:41:00.070192] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquired]
==> [2019-12-10-16:41:00.116237] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Released]
==> [2019-12-10-16:41:00.118529] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquiring]
==> [2019-12-10-16:41:00.119082] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquired]
==> [2019-12-10-16:41:00.165906] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Released]
==> [2019-12-10-16:41:00.167462] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquiring]
==> [2019-12-10-16:41:00.167934] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquired]
==> [2019-12-10-16:41:00.218965] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Released]
==> [2019-12-10-16:41:00.219271] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquiring]
==> [2019-12-10-16:41:00.219655] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquired]
==> [2019-12-10-16:41:00.266983] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Released]
==> [2019-12-10-16:41:00.267310] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquiring]
==> [2019-12-10-16:41:00.267728] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquired]
==> [2019-12-10-16:41:00.477182] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Released]
==> [2019-12-10-16:41:00.484131] WRITE LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/prefix_lock[5820083178631369124:1] [Acquiring]
==> [2019-12-10-16:41:00.485073] WRITE LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/prefix_lock[5820083178631369124:1] [Acquired]
==> [2019-12-10-16:41:00.485185] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquiring]
==> [2019-12-10-16:41:00.485539] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquired]
==> [2019-12-10-16:41:00.538750] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Released]
==> [2019-12-10-16:41:00.539044] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquiring]
==> [2019-12-10-16:41:00.539420] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquired]
==> [2019-12-10-16:41:00.585088] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Released]
==> [2019-12-10-16:41:00.588744] WRITE LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/prefix_lock[5820083178631369124:1] [Released]
==> [2019-12-10-16:41:00.598045] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/prefix_lock[5820083178631369124:1] [Acquiring]
==> [2019-12-10-16:41:00.598586] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/prefix_lock[5820083178631369124:1] [Acquired]
==> [2019-12-10-16:41:00.598719] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquiring]
==> [2019-12-10-16:41:00.599106] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquired]
==> [2019-12-10-16:41:00.645810] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Released]
==> [2019-12-10-16:41:00.646311] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/prefix_lock[5820083178631369124:1] [Released]
==> [2019-12-10-16:41:00.646450] Installing googletest dependencies
==> [2019-12-10-16:41:00.646715] [email protected] : externally installed in /opt/local
==> [2019-12-10-16:41:00.646790] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquiring]
==> [2019-12-10-16:41:00.647079] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquired]
==> [2019-12-10-16:41:00.693385] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Released]
==> [2019-12-10-16:41:00.693685] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquiring]
==> [2019-12-10-16:41:00.694108] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquired]
==> [2019-12-10-16:41:00.740314] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Released]
==> [2019-12-10-16:41:00.740884] [email protected] : generating module file
==> [2019-12-10-16:41:00.740948] SKIP: shebang filtering [external package]
==> [2019-12-10-16:41:00.741077] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquiring]
==> [2019-12-10-16:41:00.741426] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquired]
==> [2019-12-10-16:41:00.787969] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Released]
==> [2019-12-10-16:41:00.788322] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquiring]
==> [2019-12-10-16:41:00.788725] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquired]
==> [2019-12-10-16:41:00.834796] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Released]
==> [2019-12-10-16:41:00.842742] WRITE: [email protected]%[email protected]~doc+ncurses+openssl+ownlibs~qt arch=darwin-mojave-x86_64/6u2g2jh [/Users/agaspar/Code/spack/share/spack/modules/darwin-mojave-x86_64/cmake-3.15.1-clang-11.0.0-apple-6u2g2jh]
==> [2019-12-10-16:41:00.864711] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquiring]
==> [2019-12-10-16:41:00.865189] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquired]
==> [2019-12-10-16:41:00.933812] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Released]
==> [2019-12-10-16:41:01.114793] [email protected] : registering into DB
==> [2019-12-10-16:41:01.114944] WRITE LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquiring]
==> [2019-12-10-16:41:01.115530] WRITE LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquired]
==> [2019-12-10-16:41:01.164461] WRITE LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Released]
==> [2019-12-10-16:41:01.214115] Bootstrapping [email protected] compiler for googletest
==> [2019-12-10-16:41:01.214736] Installing googletest
==> [2019-12-10-16:41:01.214778] Searching for binary cache of googletest
==> [2019-12-10-16:41:01.214992] No Spack mirrors are currently configured
==> [2019-12-10-16:41:01.215405] No binary for googletest found: installing from source
==> [2019-12-10-16:41:01.215535] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquiring]
==> [2019-12-10-16:41:01.215997] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquired]
==> [2019-12-10-16:41:01.262544] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Released]
==> [2019-12-10-16:41:01.280533] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquiring]
==> [2019-12-10-16:41:01.282336] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Acquired]
==> [2019-12-10-16:41:01.533037] READ LOCK: /Users/agaspar/Code/spack/opt/spack/.spack-db/lock[0:0] [Released]
Traceback (most recent call last):
File "/Users/agaspar/Code/spack/bin/spack", line 64, in <module>
sys.exit(spack.main.main())
File "/Users/agaspar/Code/spack/lib/spack/spack/main.py", line 715, in main
return _invoke_command(command, parser, args, unknown)
File "/Users/agaspar/Code/spack/lib/spack/spack/main.py", line 457, in _invoke_command
return_val = command(parser, args)
File "/Users/agaspar/Code/spack/lib/spack/spack/cmd/install.py", line 331, in install
install_spec(args, kwargs, abstract, concrete)
File "/Users/agaspar/Code/spack/lib/spack/spack/cmd/install.py", line 209, in install_spec
spec.package.do_install(**kwargs)
File "/Users/agaspar/Code/spack/lib/spack/spack/package.py", line 1761, in do_install
PackageBase._verbose = spack.build_environment.fork(
File "/Users/agaspar/Code/spack/lib/spack/spack/build_environment.py", line 860, in fork
p.start()
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py", line 283, in _Popen
return Popen(process_obj)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
super().__init__(process_obj)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
self._launch(process_obj)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
reduction.dump(process_obj, fp)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'fork.<locals>.child_process'Information on your system
This includes:
- MacOS Mojave
- Python 3.8 from MacPorts