-
Notifications
You must be signed in to change notification settings - Fork 26.3k
Closed
Labels
high prioritymodule: flaky-testsProblem is a flaky test in CIProblem is a flaky test in CIoncall: jitAdd this issue/PR to JIT oncall triage queueAdd this issue/PR to JIT oncall triage queuetriage reviewtriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate moduleThis issue has been looked at a team member, and triaged and prioritized into an appropriate module
Description
Two back-to-back failures:
Nov 07 03:54:07 ======================================================================
Nov 07 03:54:07 ERROR: test_tracing_multiple_methods (__main__.TestScript)
Nov 07 03:54:07 ----------------------------------------------------------------------
Nov 07 03:54:07 Traceback (most recent call last):
Nov 07 03:54:07 File "test_jit.py", line 3988, in test_tracing_multiple_methods
Nov 07 03:54:07 module = torch.jit.trace_module(n, inputs)
Nov 07 03:54:07 File "/opt/python/3.5/lib/python3.5/site-packages/torch/jit/__init__.py", line 1031, in trace_module
Nov 07 03:54:07 check_tolerance, _force_outplace, True, _module_class)
Nov 07 03:54:07 File "/opt/python/3.5/lib/python3.5/site-packages/torch/autograd/grad_mode.py", line 49, in decorate_no_grad
Nov 07 03:54:07 return func(*args, **kwargs)
Nov 07 03:54:07 File "/opt/python/3.5/lib/python3.5/site-packages/torch/jit/__init__.py", line 674, in _check_trace
Nov 07 03:54:07 raise TracingCheckError(*diag_info)
Nov 07 03:54:07 torch.jit.TracingCheckError: Tracing failed sanity checks!
Nov 07 03:54:07 ERROR: Graphs differed across invocations!
Nov 07 03:54:07 Graph diff:
Nov 07 03:54:07 - graph(%self : ClassType<Module>,
Nov 07 03:54:07 + graph(%self.1 : ClassType<Module>,
Nov 07 03:54:07 ? ++
Nov 07 03:54:07 %input : Tensor):
Nov 07 03:54:07 - %2 : ClassType<Module> = prim::GetAttr[name="conv"](%self)
Nov 07 03:54:07 + %2 : ClassType<Module> = prim::GetAttr[name="conv"](%self.1)
Nov 07 03:54:07 ? ++
Nov 07 03:54:07 + %3 : Tensor = prim::CallMethod[name="forward"](%2, %input)
Nov 07 03:54:07 - %3 : Tensor = prim::GetAttr[name="bias"](%2)
Nov 07 03:54:07 - %4 : ClassType<Module> = prim::GetAttr[name="conv"](%self)
Nov 07 03:54:07 - %5 : Tensor = prim::GetAttr[name="weight"](%4)
Nov 07 03:54:07 - %6 : int = prim::Constant[value=1]() # /opt/python/3.5/lib/python3.5/site-packages/torch/nn/modules/conv.py:342:0
Nov 07 03:54:07 - %7 : int = prim::Constant[value=1]() # /opt/python/3.5/lib/python3.5/site-packages/torch/nn/modules/conv.py:342:0
Nov 07 03:54:07 - %8 : int[] = prim::ListConstruct(%6, %7)
Nov 07 03:54:07 - %9 : int = prim::Constant[value=0]() # /opt/python/3.5/lib/python3.5/site-packages/torch/nn/modules/conv.py:342:0
Nov 07 03:54:07 - %10 : int = prim::Constant[value=0]() # /opt/python/3.5/lib/python3.5/site-packages/torch/nn/modules/conv.py:342:0
Nov 07 03:54:07 - %11 : int[] = prim::ListConstruct(%9, %10)
Nov 07 03:54:07 - %12 : int = prim::Constant[value=1]() # /opt/python/3.5/lib/python3.5/site-packages/torch/nn/modules/conv.py:342:0
Nov 07 03:54:07 - %13 : int = prim::Constant[value=1]() # /opt/python/3.5/lib/python3.5/site-packages/torch/nn/modules/conv.py:342:0
Nov 07 03:54:07 - %14 : int[] = prim::ListConstruct(%12, %13)
Nov 07 03:54:07 - %15 : bool = prim::Constant[value=0]() # /opt/python/3.5/lib/python3.5/site-packages/torch/nn/modules/conv.py:342:0
Nov 07 03:54:07 - %16 : int = prim::Constant[value=0]() # /opt/python/3.5/lib/python3.5/site-packages/torch/nn/modules/conv.py:342:0
Nov 07 03:54:07 - %17 : int = prim::Constant[value=0]() # /opt/python/3.5/lib/python3.5/site-packages/torch/nn/modules/conv.py:342:0
Nov 07 03:54:07 - %18 : int[] = prim::ListConstruct(%16, %17)
Nov 07 03:54:07 - %19 : int = prim::Constant[value=1]() # /opt/python/3.5/lib/python3.5/site-packages/torch/nn/modules/conv.py:342:0
Nov 07 03:54:07 - %20 : bool = prim::Constant[value=0]() # /opt/python/3.5/lib/python3.5/site-packages/torch/nn/modules/conv.py:342:0
Nov 07 03:54:07 - %21 : bool = prim::Constant[value=0]() # /opt/python/3.5/lib/python3.5/site-packages/torch/nn/modules/conv.py:342:0
Nov 07 03:54:07 - %22 : bool = prim::Constant[value=1]() # /opt/python/3.5/lib/python3.5/site-packages/torch/nn/modules/conv.py:342:0
Nov 07 03:54:07 - %23 : Tensor = aten::_convolution(%input, %5, %3, %8, %11, %14, %15, %18, %19, %20, %21, %22) # /opt/python/3.5/lib/python3.5/site-packages/torch/nn/modules/conv.py:342:0
Nov 07 03:54:07 - return (%23)
Nov 07 03:54:07 ? -
Nov 07 03:54:07 + return (%3)
Nov 07 03:54:07 First diverging operator:
Nov 07 03:54:07 Node diff:
Nov 07 03:54:07 - %2 : ClassType<Module> = prim::GetAttr[name="conv"](%self)
Nov 07 03:54:07 + %2 : ClassType<Module> = prim::GetAttr[name="conv"](%self.1)
Nov 07 03:54:07 ? ++
Nov 07 03:54:07
Nov 07 03:54:07
Nov 07 03:54:07 ----------------------------------------------------------------------
Nov 07 03:54:07 Ran 2252 tests in 169.006s
Nov 07 03:54:07
Nov 07 03:54:07 FAILED (errors=1, skipped=61, expected failures=1)
Nov 07 03:54:07 Traceback (most recent call last):
Nov 07 03:54:07 File "test/run_test.py", line 454, in <module>
Nov 07 03:54:07 main()
Nov 07 03:54:07 File "test/run_test.py", line 446, in main
Nov 07 03:54:07 raise RuntimeError(message)
Nov 07 03:54:07 RuntimeError: test_jit failed!
Nov 07 03:54:07 + cleanup
Nov 07 03:54:07 + retcode=1
Nov 07 03:54:07 + set +x
Nov 07 03:54:07 =================== sccache compilation log ===================
@suo do you know who should take a look?
Metadata
Metadata
Assignees
Labels
high prioritymodule: flaky-testsProblem is a flaky test in CIProblem is a flaky test in CIoncall: jitAdd this issue/PR to JIT oncall triage queueAdd this issue/PR to JIT oncall triage queuetriage reviewtriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate moduleThis issue has been looked at a team member, and triaged and prioritized into an appropriate module