Skip to content

Use SHA1 hash for kernel cache key to support Linux in FIPS-compliant mode - #5988

Merged
kmaehashi merged 1 commit into
cupy:masterfrom
kmaehashi:use-sha1-hash
Nov 5, 2021
Merged

kmaehashi merged 1 commit into
cupy:masterfrom
kmaehashi:use-sha1-hash

Conversation

@kmaehashi

Copy link
Copy Markdown
Member

Closes #5959.

I chose SHA1 based on the quick micro benchmark:

>>> a = ('*' * 4096).encode()
>>> timeit('hashlib.md5(a).hexdigest()', globals=globals())
6.068204950541258
>>> timeit('hashlib.sha1(a).hexdigest()', globals=globals())
4.463794406503439
>>> timeit('hashlib.sha256(a).hexdigest()', globals=globals())
9.106148925609887

Under Python 3.6.8 on CentOS 8 stream with FIPS-compilant mode enabled:

>>> hashlib.md5(b"foo").hexdigest()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: [digital envelope routines: EVP_DigestInit_ex] disabled for FIPS

>>> hashlib.md5(b"foo", usedforsecurity=False).hexdigest()
'acbd18db4cc2f85cedef654fccc4a4d8'

>>> hashlib.sha1(b"foo").hexdigest()
'0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33'

>>> hashlib.sha1(b"foo", usedforsecurity=False).hexdigest()
'0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33'

(seems usedforsecurity is available in any versions of Python installed via RHEL/CentOS 7/8 RPM)

@kmaehashi

Copy link
Copy Markdown
Member Author

pfnCI, test this please.
(This should be very slow as kernel caches cannot be reused at all 😅 )

@kmaehashi kmaehashi added the cat:enhancement Improvements to existing features label Nov 2, 2021
@kmaehashi kmaehashi changed the title Use SHA1 hash for kernel cache key Use SHA1 hash for kernel cache key to support Linux in FIPS-compliant mode Nov 2, 2021
@kmaehashi kmaehashi added this to the v10.0.0rc1 milestone Nov 2, 2021
@leofang

leofang commented Nov 2, 2021

Copy link
Copy Markdown
Member

For some reason I can't get consistent timing with different ways of launching timeit...but looks like sha1 is a good choice

In [8]: %timeit hashlib.md5(a).hexdigest()
5.15 µs ± 1.85 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

In [9]: %timeit hashlib.sha256(a).hexdigest()
2.29 µs ± 1.05 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

In [10]: %timeit hashlib.sha1(a).hexdigest()
2.15 µs ± 0.303 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

In [11]: timeit('hashlib.md5(a).hexdigest()')
4.6 ns ± 0.0011 ns per loop (mean ± std. dev. of 7 runs, 100000000 loops each)

In [12]: timeit('hashlib.sha1(a).hexdigest()')
4.61 ns ± 0.0055 ns per loop (mean ± std. dev. of 7 runs, 100000000 loops each)

In [13]: timeit('hashlib.sha256(a).hexdigest()')
4.79 ns ± 0.396 ns per loop (mean ± std. dev. of 7 runs, 100000000 loops each)

@chainer-ci

Copy link
Copy Markdown
Member

Jenkins CI test (for commit a3cf5d6, target branch master) failed with status FAILURE.

@kmaehashi

Copy link
Copy Markdown
Member Author

All test failures are timeout, I believe they're harmless.

@emcastillo emcastillo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@emcastillo

Copy link
Copy Markdown
Member

Jenkins, test this please

@emcastillo
emcastillo enabled auto-merge November 4, 2021 01:51
@emcastillo emcastillo self-assigned this Nov 4, 2021
@chainer-ci

Copy link
Copy Markdown
Member

Jenkins CI test (for commit a3cf5d6, target branch master) succeeded!

@kmaehashi

Copy link
Copy Markdown
Member Author

Test failure are timeout and known one (#5678).

@kmaehashi
kmaehashi disabled auto-merge November 5, 2021 02:44
@kmaehashi

Copy link
Copy Markdown
Member Author

Merging as this is review approved and test failures are unrelated to this PR.

@kmaehashi
kmaehashi merged commit c6ed9fd into cupy:master Nov 5, 2021
@kmaehashi
kmaehashi deleted the use-sha1-hash branch November 7, 2021 07:39
@leofang

leofang commented Nov 23, 2021

Copy link
Copy Markdown
Member

@jch1996 Those were fixed in #6023.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cat:enhancement Improvements to existing features prio:medium

Projects

None yet

Development

Successfully merging this pull request may close these issues.

md5 disabled for FIPS

4 participants