CRFTestCaseCuda test case issue #2023
Description
Describe the bug
one of the test cases of tests.test_crf_cuda.CRFTestCaseCuda fails randomly (with a small chance)
To Reproduce
this error
e.g. CI log: https://github.com/Project-MONAI/MONAI/runs/2361733732?check_suite_focus=true#step:6:7821
======================================================================
FAIL: test_2_1_batche_s_3_dimension_s_2_classe_s_1_channel_s_ (tests.test_crf_cuda.CRFTestCaseCuda)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/parameterized/parameterized.py", line 533, in standalone_func
return func(*(a + p.args), **p.kwargs)
File "/__w/MONAI/MONAI/tests/test_crf_cuda.py", line 447, in test
np.testing.assert_allclose(output, expected, atol=1e-3, rtol=1e-3)
File "/usr/local/lib/python3.6/dist-packages/numpy/testing/_private/utils.py", line 1528, in assert_allclose
verbose=verbose, header=header, equal_nan=equal_nan)
File "/usr/local/lib/python3.6/dist-packages/numpy/testing/_private/utils.py", line 840, in assert_array_compare
raise AssertionError(msg)
AssertionError:
Not equal to tolerance rtol=0.001, atol=0.001
Mismatched elements: 63 / 250 (25.2%)
Max absolute difference: 0.021
Max relative difference: 0.114
Expected behavior
no failure for this test case
cc @charliebudd