Python 3.9.6: Error calling urllib.parse #24644
Description
With Python 3.9.6, this change in urllib.parse causes an error using urllib.parse.urlparse(..., scheme=None) like here:
spack/lib/spack/spack/util/url.py
Lines 167 to 168 in 59028aa
I don't think this is a Python bug since the docs state:
[scheme] should be the same type (text or bytes) as urlstring
However, typeshed does have it typed as Optional[str] so I'm not sure.
Steps to reproduce the issue
$ python3 --version
Python 3.9.6
$ spack install zlib
==> Installing zlib-1.2.11-74mwnxgn6nujehpyyalhwizwojwn5zga
==> Error: Failed to install zlib due to AttributeError: 'NoneType' object has no attribute 'replace'
==> Error: 'NoneType' object has no attribute 'replace'
$ python3 -c "from urllib.parse import urlparse; urlparse('', scheme=None)"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/local/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/parse.py", line 393, in urlparse
splitresult = urlsplit(url, scheme, allow_fragments)
File "/usr/local/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/parse.py", line 462, in urlsplit
scheme = scheme.replace(b, "")
AttributeError: 'NoneType' object has no attribute 'replace'Error Message
Stacktrace
$ spack --debug --stacktrace install zlib
Cellar/spack/0.16.2/lib/spack/spack/cmd/__init__.py:122 ==> [2021-06-30-19:23:27.332263] Imported install from built-in commands
Cellar/spack/0.16.2/lib/spack/spack/config.py:905 ==> [2021-06-30-19:23:27.336600] Reading config file /usr/local/Cellar/spack/0.16.2/etc/spack/defaults/config.yaml
Cellar/spack/0.16.2/lib/spack/spack/cmd/__init__.py:122 ==> [2021-06-30-19:23:27.370003] Imported install from built-in commands
Cellar/spack/0.16.2/lib/spack/spack/config.py:905 ==> [2021-06-30-19:23:27.379310] Reading config file /usr/local/Cellar/spack/0.16.2/etc/spack/defaults/repos.yaml
Cellar/spack/0.16.2/lib/spack/spack/config.py:905 ==> [2021-06-30-19:23:27.431903] Reading config file /usr/local/Cellar/spack/0.16.2/etc/spack/defaults/packages.yaml
Cellar/spack/0.16.2/lib/spack/spack/config.py:905 ==> [2021-06-30-19:23:27.463903] Reading config file /usr/local/Cellar/spack/0.16.2/etc/spack/defaults/darwin/packages.yaml
Cellar/spack/0.16.2/lib/spack/spack/config.py:905 ==> [2021-06-30-19:23:27.473058] Reading config file /Users/Branch/.spack/darwin/compilers.yaml
Cellar/spack/0.16.2/lib/spack/spack/database.py:361 ==> [2021-06-30-19:23:27.525082] DATABASE LOCK TIMEOUT: 3s
Cellar/spack/0.16.2/lib/spack/spack/database.py:365 ==> [2021-06-30-19:23:27.525475] PACKAGE LOCK TIMEOUT: No timeout
Cellar/spack/0.16.2/lib/spack/spack/installer.py:1371 ==> [2021-06-30-19:23:27.532093] Initializing the build queue from the build requests
Cellar/spack/0.16.2/lib/spack/spack/installer.py:989 ==> [2021-06-30-19:23:27.532547] Initializing the build queue for zlib
Cellar/spack/0.16.2/lib/spack/spack/installer.py:2054 ==> [2021-06-30-19:23:27.533780] Processing dependencies for zlib-1.2.11-74mwnxgn6nujehpyyalhwizwojwn5zga: ('build', 'link', 'run')
Cellar/spack/0.16.2/lib/spack/spack/database.py:461 ==> [2021-06-30-19:23:27.553180] Removing failure marking for zlib
Cellar/spack/0.16.2/lib/spack/spack/installer.py:1793 ==> [2021-06-30-19:23:27.553894] Pkg id zlib-1.2.11-74mwnxgn6nujehpyyalhwizwojwn5zga has the following dependents:
Cellar/spack/0.16.2/lib/spack/spack/installer.py:1377 ==> [2021-06-30-19:23:27.555060] Ensure all dependencies know all dependents across specs
Cellar/spack/0.16.2/lib/spack/spack/installer.py:935 ==> [2021-06-30-19:23:27.556576] Acquiring a write lock on zlib-1.2.11-74mwnxgn6nujehpyyalhwizwojwn5zga with timeout 1e-09
Cellar/spack/0.16.2/lib/spack/spack/stage.py:312 ==> [2021-06-30-19:23:27.561416] Creating stage lock spack-stage-zlib-1.2.11-74mwnxgn6nujehpyyalhwizwojwn5zga
Cellar/spack/0.16.2/lib/spack/spack/installer.py:1095 ==> [2021-06-30-19:23:27.563466] Installing zlib-1.2.11-74mwnxgn6nujehpyyalhwizwojwn5zga
Cellar/spack/0.16.2/lib/spack/spack/installer.py:393 ==> [2021-06-30-19:23:27.563803] Searching for binary cache of zlib-1.2.11-74mwnxgn6nujehpyyalhwizwojwn5zga
Cellar/spack/0.16.2/lib/spack/spack/config.py:905 ==> [2021-06-30-19:23:27.565923] Reading config file /usr/local/Cellar/spack/0.16.2/etc/spack/defaults/mirrors.yaml
Cellar/spack/0.16.2/lib/spack/spack/installer.py:1301 ==> [2021-06-30-19:23:27.569783] Flagging zlib-1.2.11-74mwnxgn6nujehpyyalhwizwojwn5zga as failed: 'NoneType' object has no attribute 'replace'
Cellar/spack/0.16.2/lib/spack/spack/installer.py:1577 ==> [2021-06-30-19:23:27.571197] Error: Failed to install zlib due to AttributeError: 'NoneType' object has no attribute 'replace'
Traceback (most recent call last):
File "/usr/local/bin/spack", line 66, in <module>
sys.exit(spack.main.main())
File "/usr/local/Cellar/spack/0.16.2/lib/spack/spack/main.py", line 762, in main
return _invoke_command(command, parser, args, unknown)
File "/usr/local/Cellar/spack/0.16.2/lib/spack/spack/main.py", line 490, in _invoke_command
return_val = command(parser, args)
File "/usr/local/Cellar/spack/0.16.2/lib/spack/spack/cmd/install.py", line 365, in install
install_specs(args, kwargs, zip(abstract_specs, specs))
File "/usr/local/Cellar/spack/0.16.2/lib/spack/spack/cmd/install.py", line 207, in install_specs
builder.install()
File "/usr/local/Cellar/spack/0.16.2/lib/spack/spack/installer.py", line 1535, in install
self._install_task(task)
File "/usr/local/Cellar/spack/0.16.2/lib/spack/spack/installer.py", line 1101, in _install_task
_install_from_cache(pkg, cache_only, explicit, unsigned,
File "/usr/local/Cellar/spack/0.16.2/lib/spack/spack/installer.py", line 273, in _install_from_cache
installed_from_cache = _try_install_from_binary_cache(
File "/usr/local/Cellar/spack/0.16.2/lib/spack/spack/installer.py", line 394, in _try_install_from_binary_cache
matches = binary_distribution.get_mirrors_for_spec(
File "/usr/local/Cellar/spack/0.16.2/lib/spack/spack/binary_distribution.py", line 1396, in get_mirrors_for_spec
results = try_direct_fetch(spec,
File "/usr/local/Cellar/spack/0.16.2/lib/spack/spack/binary_distribution.py", line 1336, in try_direct_fetch
buildcache_fetch_url = url_util.join(
File "/usr/local/Cellar/spack/0.16.2/lib/spack/spack/util/url.py", line 156, in join
obj = urllib_parse.urlparse(
File "/usr/local/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/parse.py", line 393, in urlparse
splitresult = urlsplit(url, scheme, allow_fragments)
File "/usr/local/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/parse.py", line 462, in urlsplit
scheme = scheme.replace(b, "")
AttributeError: 'NoneType' object has no attribute 'replace'Information on your system
- Spack: 0.16.2
- Python: 3.9.6
- Platform: darwin-bigsur-skylake
Additional information
- I have run
spack debug reportand reported the version of Spack/Python/Platform - I have searched the issues of this repo and believe this is not a duplicate
- I have run the failing commands in debug mode and reported the output