Repository navigation
Description
Context
Python compiled for FIPS (built with FIPS compatible openssl) does not allow the use of the md5 algorithm. Typically, md5 is completely disabled by a custom patch to Python to prevent the usage of md5 for FIPS compliance, as Python may fallback to its own md5 implementation if openssl does not provide one, which is true for FIPS.
The import of md5 from the hashlib module is problematic because urllib is widely used in Python, such as when installing pip and this completely breaks down when FIPS mode is enabled.
https://github.com/urllib3/urllib3/blob/main/src/urllib3/util/ssl_.py
$ python3 -m ensurepip --upgrade --default-pip
...
File "/tmp/tmp052h8i17/pip-24.0-py3-none-any.whl/pip/_vendor/requests/__init__.py", line 43, in <module>
File "/tmp/tmp052h8i17/pip-24.0-py3-none-any.whl/pip/_vendor/urllib3/__init__.py", line 13, in <module>
File "/tmp/tmp052h8i17/pip-24.0-py3-none-any.whl/pip/_vendor/urllib3/connectionpool.py", line 12, in <module>
File "/tmp/tmp052h8i17/pip-24.0-py3-none-any.whl/pip/_vendor/urllib3/connection.py", line 15, in <module>
File "/tmp/tmp052h8i17/pip-24.0-py3-none-any.whl/pip/_vendor/urllib3/util/__init__.py", line 8, in <module>
File "/tmp/tmp052h8i17/pip-24.0-py3-none-any.whl/pip/_vendor/urllib3/util/ssl_.py", line 8, in <module>
ImportError: cannot import name 'md5' from 'hashlib' (/opt/python-fips/lib/python3.12/hashlib.py)
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/opt/python-fips/lib/python3.12/ensurepip/__main__.py", line 5, in <module>
sys.exit(ensurepip._main())
^^^^^^^^^^^^^^^^^
File "/opt/python-fips/lib/python3.12/ensurepip/__init__.py", line 284, in _main
return _bootstrap(
^^^^^^^^^^^
File "/opt/python-fips/lib/python3.12/ensurepip/__init__.py", line 200, in _bootstrap
return _run_pip([*args, *_PACKAGE_NAMES], additional_paths)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/python-fips/lib/python3.12/ensurepip/__init__.py", line 101, in _run_pip
return subprocess.run(cmd, check=True).returncode
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/python-fips/lib/python3.12/subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/local/bin/python3', '-W', 'ignore::DeprecationWarning', '-c', '\nimport runpy\nimport sys\nsys.path = [\'/tmp/tmp052h8i17/pip-24.0-py3-none-any.whl\'] + sys.path\nsys.argv[1:] = [\'install\', \'--no-cache-dir\', \'--no-index\', \'--find-links\', \'/tmp/tmp052h8i17\', \'--upgrade\', \'pip\']\n
As a result, the urllib module is unusable with Python compiled for FIPS.
Would it be possible to provide a way to get around this problem, given that in such an environment I would like to disable the usage of md5?
Activity
sethmlarson commentedon Jul 29, 2024
Thanks for reporting this! We should make our usage of hashes lazy so that removed hashes don't block urllib3's usage in FIPS mode (or other scenarios where hashes are removed).
illia-v commentedon Jul 29, 2024
A fix will need to end up in 1.26.x to be available for pip. And one more option is creating
HASHFUNC_MAPdynamicallyThis should be more backwards compatible since
HASHFUNC_MAPis kind of public APIWe'll need a different error message for cases when a fingerprint is of a valid length, but a needed hash function is absent
urllib3/src/urllib3/util/ssl_.py
Lines 162 to 164 in 07f236b
sethmlarson commentedon Jul 29, 2024
Agreed @illia-v, since this is a non-trivial amount of work I've attached a bounty. Fix must go into both 2.x and 1.26.x
shashankram commentedon Jul 30, 2024
Besides the changes noted in #3432 (comment), is there additional work involved?
sethmlarson commentedon Jul 30, 2024
@shashankram I don't think so, all that's needed is what's in the issue already :)
shashankram commentedon Jul 30, 2024
@sethmlarson I can help with this!
util/ssl: make code resilient to missing hash functions
util/ssl: make code resilient to missing hash functions
util/ssl: make code resilient to missing hash functions
Fix a crash where certain hash functions are absent (#3434)
Dependency Updates: Bump urllib3 from 1.26.19 to 1.26.20 in /script/l…
16 remaining items
deps: update dependency urllib3 to v2.2.3 (#3194)
[Dependency] Bump urllib3 from 2.2.2 to 2.2.3 in /.setup/pip (#10993)
chore(deps): bump urllib3 from 2.2.2 to 2.2.3 (#208)
build(deps): bump urllib3 from 2.2.2 to 2.2.3 in /docs (#2301)
Bump the pip group across 1 directory with 4 updates (#68)
chore(deps): bump urllib3 from 2.2.2 to 2.2.3 (#6492)
Bump urllib3 from 2.2.2 to 2.3.0 (#15)
chore(deps): Bump urllib3 from 2.2.2 to 2.3.0 in /experimenter/tests (#…
Bump urllib3 from 1.26.20 to 2.3.0 (#199)
Bump urllib3 from 1.26.20 to 2.3.0 (#207)
Bump urllib3 from 1.26.20 to 2.3.0 (#216)
Bump urllib3 from 1.26.20 to 2.3.0 (#220)
Bump urllib3 from 1.26.20 to 2.3.0 (#232)
Bump urllib3 from 1.26.20 to 2.3.0 (#239)
Bump urllib3 from 1.26.20 to 2.3.0 (#246)