This repository was archived by the owner on Nov 17, 2023. It is now read-only.
This repository was archived by the owner on Nov 17, 2023. It is now read-only.
test_operator_gpu.test_rms fails #8230
Description
Description
test_operator_gpu.test_rms fails causing build failure.
Failed build:
https://builds.apache.org/view/Incubator%20Projects/job/incubator-mxnet/job/master/503/
https://builds.apache.org/blue/organizations/jenkins/incubator-mxnet/detail/master/514/pipeline
Error log:
[Python3: MKLML-GPU] ======================================================================
[Python3: MKLML-GPU] FAIL: test_operator_gpu.test_rms
[Python3: MKLML-GPU] ----------------------------------------------------------------------
[Python3: MKLML-GPU] Traceback (most recent call last):
[Python3: MKLML-GPU] File "/usr/local/lib/python3.4/dist-packages/nose/case.py", line 198, in runTest
[Python3: MKLML-GPU] self.test(*self.arg)
[Python3: MKLML-GPU] File "/usr/local/lib/python3.4/dist-packages/nose/util.py", line 620, in newfunc
[Python3: MKLML-GPU] return func(*arg, **kw)
[Python3: MKLML-GPU] File "/workspace/tests/python/gpu/../unittest/test_optimizer.py", line 564, in test_rms
[Python3: MKLML-GPU] compare_optimizer(opt1(**kwarg), opt2(**kwarg), shape, dtype)
[Python3: MKLML-GPU] File "/workspace/tests/python/gpu/../unittest/test_optimizer.py", line 101, in compare_optimizer
[Python3: MKLML-GPU] compare_ndarray_tuple(state1, state2, rtol=1e-4, atol=1e-5)
[Python3: MKLML-GPU] File "/workspace/tests/python/gpu/../unittest/test_optimizer.py", line 72, in compare_ndarray_tuple
[Python3: MKLML-GPU] compare_ndarray_tuple(s1, s2, rtol, atol)
[Python3: MKLML-GPU] File "/workspace/tests/python/gpu/../unittest/test_optimizer.py", line 74, in compare_ndarray_tuple
[Python3: MKLML-GPU] assert_almost_equal(t1.asnumpy(), t2.asnumpy(), rtol=rtol, atol=atol)
[Python3: MKLML-GPU] File "/workspace/python/mxnet/test_utils.py", line 491, in assert_almost_equal
[Python3: MKLML-GPU] raise AssertionError(msg)
[Python3: MKLML-GPU] AssertionError:
[Python3: MKLML-GPU] Items are not equal:
[Python3: MKLML-GPU] Error 319.562256 exceeds tolerance rtol=0.000100, atol=0.000010. Location of maximum error:(0, 0, 2), a=0.007485, b=0.004156
[Python3: MKLML-GPU] a: array([[[ 0.016 , 0.016 , 0.00748458, 0.016 , 0.016 ],
[Python3: MKLML-GPU] [ 0.01204856, 0.016 , 0.00497273, 0.016 , 0.016 ],
[Python3: MKLML-GPU] [ 0.00171078, 0.016 , 0.016 , 0.016 , 0.016 ],...
[Python3: MKLML-GPU] b: array([[[ 0.016 , 0.016 , 0.00415614, 0.016 , 0.016 ],
[Python3: MKLML-GPU] [ 0.00951787, 0.016 , 0.00399938, 0.016 , 0.016 ],
[Python3: MKLML-GPU] [ 0.00043585, 0.016 , 0.01412016, 0.016 , 0.016 ],...
[Python3: MKLML-GPU]
The failing code is from PR #7654.
Environment info (Required)
Failed on the build machine.
Build config: Python3: MKLML-GPU
Check https://builds.apache.org/blue/organizations/jenkins/incubator-mxnet/detail/master/503/pipeline
MXNet commit hash:
2ff9584
Steps to reproduce
Build and run unit test. See above for more details.