🐛 Describe the bug

pytorch2.0 , add model = torch.compile(model) to fairseq on
https://github.com/facebookresearch/fairseq/blob/main/fairseq_cli/train.py#L97

and train hubert model, got an error

torch._dynamo.exc.BackendCompilerFailed: debug_wrapper raised DynamicOutputShapeException: aten.index.Tensor

Error logs

File "/tools/fairseq/fairseq2/fairseq/fairseq/models/hubert/hubert.py", line 352, in
mask_indices = torch.from_numpy(mask_indices).to(x.device)
File "/tools/anaconda/envs/myenv/lib/python3.9/site-packages/torch/_dynamo/eval_frame.py", line 337, in catch_errors
return callback(frame, cache_size, hooks)
File "/tools/anaconda/envs/myenv/lib/python3.9/site-packages/torch/_dynamo/convert_frame.py", line 404, in _convert_frame
result = inner_convert(frame, cache_size, hooks)
File "/tools/anaconda/envs/myenv/lib/python3.9/site-packages/torch/_dynamo/convert_frame.py", line 104, in _fn
return fn(*args, **kwargs)
File "/tools/anaconda/envs/myenv/lib/python3.9/site-packages/torch/_dynamo/convert_frame.py", line 262, in _convert_frame_assert
return _compile(
File "/tools/anaconda/envs/myenv/lib/python3.9/site-packages/torch/_dynamo/utils.py", line 163, in time_wrapper
r = func(*args, **kwargs)
File "/tools/anaconda/envs/myenv/lib/python3.9/site-packages/torch/_dynamo/convert_frame.py", line 324, in _compile
out_code = transform_code_object(code, transform)
File "/tools/anaconda/envs/myenv/lib/python3.9/site-packages/torch/_dynamo/bytecode_transformation.py", line 445, in transform_code_object
transformations(instructions, code_options)
File "/tools/anaconda/envs/myenv/lib/python3.9/site-packages/torch/_dynamo/convert_frame.py", line 311, in transform
tracer.run()
File "/tools/anaconda/envs/myenv/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 1726, in run
super().run()
File "/tools/anaconda/envs/myenv/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 576, in run
and self.step()
File "/tools/anaconda/envs/myenv/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 540, in step
getattr(self, inst.opname)(inst)
File "/tools/anaconda/envs/myenv/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 1792, in RETURN_VALUE
self.output.compile_subgraph(
File "/tools/anaconda/envs/myenv/lib/python3.9/site-packages/torch/_dynamo/output_graph.py", line 541, in compile_subgraph
self.compile_and_call_fx_graph(tx, pass2.graph_output_vars(), root)
File "/tools/anaconda/envs/myenv/lib/python3.9/site-packages/torch/_dynamo/output_graph.py", line 588, in compile_and_call_fx_graph
compiled_fn = self.call_user_compiler(gm)
File "/tools/anaconda/envs/myenv/lib/python3.9/site-packages/torch/_dynamo/utils.py", line 163, in time_wrapper
r = func(*args, **kwargs)
File "/tools/anaconda/envs/myenv/lib/python3.9/site-packages/torch/_dynamo/output_graph.py", line 675, in call_user_compiler
raise BackendCompilerFailed(self.compiler_fn, e) from e
torch._dynamo.exc.BackendCompilerFailed: debug_wrapper raised DynamicOutputShapeException: aten.index.Tensor

Minified repro

No response

Versions

pytorch2.0

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 @soumith @ngimel