Skip to content

Usage of md5 in util/ssl_.py breaks in FIPS mode #3432

Closed

Description

@shashankram

Context

Python compiled for FIPS (built with FIPS compatible openssl) does not allow the use of the md5 algorithm. Typically, md5 is completely disabled by a custom patch to Python to prevent the usage of md5 for FIPS compliance, as Python may fallback to its own md5 implementation if openssl does not provide one, which is true for FIPS.

The import of md5 from the hashlib module is problematic because urllib is widely used in Python, such as when installing pip and this completely breaks down when FIPS mode is enabled.
https://github.com/urllib3/urllib3/blob/main/src/urllib3/util/ssl_.py

$ python3 -m ensurepip --upgrade --default-pip
...
  File "/tmp/tmp052h8i17/pip-24.0-py3-none-any.whl/pip/_vendor/requests/__init__.py", line 43, in <module>
  File "/tmp/tmp052h8i17/pip-24.0-py3-none-any.whl/pip/_vendor/urllib3/__init__.py", line 13, in <module>
  File "/tmp/tmp052h8i17/pip-24.0-py3-none-any.whl/pip/_vendor/urllib3/connectionpool.py", line 12, in <module>
  File "/tmp/tmp052h8i17/pip-24.0-py3-none-any.whl/pip/_vendor/urllib3/connection.py", line 15, in <module>
  File "/tmp/tmp052h8i17/pip-24.0-py3-none-any.whl/pip/_vendor/urllib3/util/__init__.py", line 8, in <module>
  File "/tmp/tmp052h8i17/pip-24.0-py3-none-any.whl/pip/_vendor/urllib3/util/ssl_.py", line 8, in <module>
ImportError: cannot import name 'md5' from 'hashlib' (/opt/python-fips/lib/python3.12/hashlib.py)
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/opt/python-fips/lib/python3.12/ensurepip/__main__.py", line 5, in <module>
    sys.exit(ensurepip._main())
             ^^^^^^^^^^^^^^^^^
  File "/opt/python-fips/lib/python3.12/ensurepip/__init__.py", line 284, in _main
    return _bootstrap(
           ^^^^^^^^^^^
  File "/opt/python-fips/lib/python3.12/ensurepip/__init__.py", line 200, in _bootstrap
    return _run_pip([*args, *_PACKAGE_NAMES], additional_paths)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/python-fips/lib/python3.12/ensurepip/__init__.py", line 101, in _run_pip
    return subprocess.run(cmd, check=True).returncode
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/python-fips/lib/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/local/bin/python3', '-W', 'ignore::DeprecationWarning', '-c', '\nimport runpy\nimport sys\nsys.path = [\'/tmp/tmp052h8i17/pip-24.0-py3-none-any.whl\'] + sys.path\nsys.argv[1:] = [\'install\', \'--no-cache-dir\', \'--no-index\', \'--find-links\', \'/tmp/tmp052h8i17\', \'--upgrade\', \'pip\']\n

As a result, the urllib module is unusable with Python compiled for FIPS.

Would it be possible to provide a way to get around this problem, given that in such an environment I would like to disable the usage of md5?

Activity

  1. sethmlarson commented on Jul 29, 2024

    @sethmlarson
    Member

    Thanks for reporting this! We should make our usage of hashes lazy so that removed hashes don't block urllib3's usage in FIPS mode (or other scenarios where hashes are removed).

  2. illia-v commented on Jul 29, 2024

    @illia-v
    Member

    A fix will need to end up in 1.26.x to be available for pip. And one more option is creating HASHFUNC_MAP dynamically

    HASHFUNC_MAP = {length: getattr(hashlib, alg, None) for length, alg in ((32, "md5"), (40, "sha1"), (64, "sha256"))}

    This should be more backwards compatible since HASHFUNC_MAP is kind of public API

    We'll need a different error message for cases when a fingerprint is of a valid length, but a needed hash function is absent

    hashfunc = HASHFUNC_MAP.get(digest_length)
    if not hashfunc:
    raise SSLError(f"Fingerprint of invalid length: {fingerprint}")

  3. added
    💰 Bounty $100If you complete this issue we'll pay you $100 on OpenCollective!
    on Jul 29, 2024
  4. sethmlarson commented on Jul 29, 2024

    @sethmlarson
    Member

    Agreed @illia-v, since this is a non-trivial amount of work I've attached a bounty. Fix must go into both 2.x and 1.26.x

  5. shashankram commented on Jul 30, 2024

    @shashankram
    ContributorAuthor

    Besides the changes noted in #3432 (comment), is there additional work involved?

  6. sethmlarson commented on Jul 30, 2024

    @sethmlarson
    Member

    @shashankram I don't think so, all that's needed is what's in the issue already :)

  7. shashankram commented on Jul 30, 2024

    @shashankram
    ContributorAuthor

    @sethmlarson I can help with this!

  8. 16 remaining items

  9. added a commit that references this issue on Dec 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions