[ONNX] Enable opset 13 ORT tests by neginraoof · Pull Request #46903 · pytorch/pytorch

neginraoof · 2020-10-27T06:45:50Z

Enable export of opset 13 ops.

dr-ci · 2020-10-27T06:58:05Z

💊 CI failures summary and remediations

As of commit 83272f8 (more details on the Dr. CI page):

7/8 failures possibly* introduced in this PR
- 4/7 non-CircleCI failure(s)
1/8 broken upstream at merge base 5912316 on Dec 15 from 12:16pm to 1:44pm PDT (6 commits; 40a02e2 - f4e15c4)

🕵️ 3 new failures recognized by patterns

The following CI failures do not appear to be due to upstream breakages (reran 1 job to discount flakiness):

pytorch_linux_xenial_cuda9_2_cudnn7_py3_gcc5_4_build (1/3)

Step: "(Optional) Merge target branch" (full log | diagnosis details | 🔁 rerun) <confirmed not flaky by 2 failures>

Automatic merge failed; fix conflicts and then commit the result.

CONFLICT (add/add): Merge conflict in CONTRIBUTING.md 
Auto-merging CONTRIBUTING.md 
CONFLICT (add/add): Merge conflict in .jenkins/pytorch/win-build.sh 
Auto-merging .jenkins/pytorch/win-build.sh 
CONFLICT (add/add): Merge conflict in .github/workflows/lint.yml 
Auto-merging .github/workflows/lint.yml 
CONFLICT (add/add): Merge conflict in .clang-tidy 
Auto-merging .clang-tidy 
CONFLICT (add/add): Merge conflict in .circleci/scripts/binary_linux_test.sh 
Auto-merging .circleci/scripts/binary_linux_test.sh 
Automatic merge failed; fix conflicts and then commit the result.

pytorch_linux_xenial_py3_6_gcc5_4_build (2/3)

Step: "(Optional) Merge target branch" (full log | diagnosis details | 🔁 rerun) <confirmed not flaky by 2 failures>

Automatic merge failed; fix conflicts and then commit the result.

CONFLICT (add/add): Merge conflict in CONTRIBUTING.md 
Auto-merging CONTRIBUTING.md 
CONFLICT (add/add): Merge conflict in .jenkins/pytorch/win-build.sh 
Auto-merging .jenkins/pytorch/win-build.sh 
CONFLICT (add/add): Merge conflict in .github/workflows/lint.yml 
Auto-merging .github/workflows/lint.yml 
CONFLICT (add/add): Merge conflict in .clang-tidy 
Auto-merging .clang-tidy 
CONFLICT (add/add): Merge conflict in .circleci/scripts/binary_linux_test.sh 
Auto-merging .circleci/scripts/binary_linux_test.sh 
Automatic merge failed; fix conflicts and then commit the result.

pytorch_linux_xenial_py3_clang7_onnx_ort_test2 (3/3)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun) <confirmed not flaky by 2 failures>

Dec 18 21:36:15 test/onnx/test_pytorch_onnx_onnxruntime.py::TestONNXRuntime_opset13::test_prelu Fatal Python error: Segmentation fault

Dec 18 21:36:11 test/onnx/test_pytorch_onnx_onnxruntime.py::TestONNXRuntime_opset13::test_numel_empty FAILED [ 77%] 
Dec 18 21:36:11 test/onnx/test_pytorch_onnx_onnxruntime.py::TestONNXRuntime_opset13::test_one_hot PASSED [ 77%] 
Dec 18 21:36:11 test/onnx/test_pytorch_onnx_onnxruntime.py::TestONNXRuntime_opset13::test_ones_bool SKIPPED [ 78%] 
Dec 18 21:36:11 test/onnx/test_pytorch_onnx_onnxruntime.py::TestONNXRuntime_opset13::test_onnx_proto_checker PASSED [ 78%] 
Dec 18 21:36:11 test/onnx/test_pytorch_onnx_onnxruntime.py::TestONNXRuntime_opset13::test_optional_inputs_with_all_optionals PASSED [ 78%] 
Dec 18 21:36:12 test/onnx/test_pytorch_onnx_onnxruntime.py::TestONNXRuntime_opset13::test_optional_inputs_with_mixed_optionals FAILED [ 78%] 
Dec 18 21:36:12 test/onnx/test_pytorch_onnx_onnxruntime.py::TestONNXRuntime_opset13::test_optional_inputs_with_no_optionals PASSED [ 78%] 
Dec 18 21:36:12 test/onnx/test_pytorch_onnx_onnxruntime.py::TestONNXRuntime_opset13::test_pad_types SKIPPED [ 79%] 
Dec 18 21:36:13 test/onnx/test_pytorch_onnx_onnxruntime.py::TestONNXRuntime_opset13::test_pixel_shuffle FAILED [ 79%] 
Dec 18 21:36:14 test/onnx/test_pytorch_onnx_onnxruntime.py::TestONNXRuntime_opset13::test_pow FAILED [ 79%] 
Dec 18 21:36:15 test/onnx/test_pytorch_onnx_onnxruntime.py::TestONNXRuntime_opset13::test_prelu Fatal Python error: Segmentation fault 
Dec 18 21:36:15  
Dec 18 21:36:15 Current thread 0x00007f69928eb700 (most recent call first): 
Dec 18 21:36:15   File "/opt/conda/lib/python3.6/site-packages/_pytest/_io/saferepr.py", line 58 in repr_instance 
Dec 18 21:36:15   File "/opt/conda/lib/python3.6/reprlib.py", line 65 in repr1 
Dec 18 21:36:15   File "/opt/conda/lib/python3.6/reprlib.py", line 55 in repr 
Dec 18 21:36:15   File "/opt/conda/lib/python3.6/site-packages/_pytest/_io/saferepr.py", line 49 in repr 
Dec 18 21:36:15   File "/opt/conda/lib/python3.6/site-packages/_pytest/_io/saferepr.py", line 88 in saferepr 
Dec 18 21:36:15   File "/opt/conda/lib/python3.6/site-packages/_pytest/_code/code.py", line 686 in repr_args 
Dec 18 21:36:15   File "/opt/conda/lib/python3.6/site-packages/_pytest/_code/code.py", line 776 in repr_traceback_entry 
Dec 18 21:36:15   File "/opt/conda/lib/python3.6/site-packages/_pytest/_code/code.py", line 825 in repr_traceback

1 failure confirmed as flaky and can be ignored:

pytorch_windows_vs2019_py36_cuda10.1_test1

🚧 1 fixed upstream failure:

These were probably caused by upstream breakages that were already fixed.

Please rebase on the viable/strict branch (expand for instructions)

Since your merge base is older than viable/strict, run these commands:

git fetch https://github.com/pytorch/pytorch viable/strict
git rebase FETCH_HEAD

Check out the recency history of this "viable master" tracking branch.

pytorch_xla_linux_bionic_py3_6_clang9_build on Dec 15 from 12:16pm to 1:44pm PDT (6 commits; 40a02e2 - f4e15c4)
- 🔁 rerun

Extra GitHub checks: 4 failed

Failed: GitHub Actions - flake8-py3
Failed: GitHub Actions - quick-checks
Failed: GitHub Actions - clang-tidy
Failed: GitHub Actions - cmakelint

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group.

See how this bot performed.

This comment has been revised 63 times.

…neraoof/opset13

…oof/opset13 # Conflicts: # .jenkins/caffe2/test.sh

…oof/opset13

BowenBao

Looks good, thanks! Some minor comments

scripts/onnx/test.sh

BowenBao · 2020-12-03T18:56:56Z

test/onnx/test_pytorch_onnx_onnxruntime.py

        if torch.cuda.is_available():
            torch.cuda.manual_seed_all(0)
        np.random.seed(seed=0)
+        os.environ['ALLOW_RELEASED_ONNX_OPSET_ONLY'] = '0'


curious why we need this?

This is to enable opset 13 in ORT before official release.

BowenBao · 2020-12-03T18:58:24Z

test/onnx/test_pytorch_onnx_onnxruntime.py


+# opset 13 tests, with keep_initializers_as_inputs=False for
+# IR version 4 style export.
+TestONNXRuntime_opset13_IRv4 = type(str("TestONNXRuntime_opset13_IRv4"),


actually not sure if there is still the need of separate tests on IR versions, since sequence proto requires IR version > 4 anyways ...

BowenBao · 2020-12-03T18:59:09Z

torch/onnx/symbolic_helper.py


 _default_onnx_opset_version = 9
-_onnx_master_opset = 10
+_onnx_main_opset = 13


Thanks for fixing this!

BowenBao · 2020-12-03T19:00:42Z

torch/onnx/symbolic_opset13.py

+import torch.onnx.symbolic_helper as sym_help
+from torch.onnx.symbolic_helper import parse_args, _unimplemented
+
+block_listed_operators = ['embedding_bag']


is it due to loop that embedding_bag not supported?

Yes, exactly

)

Enable opset 13 ORT tests

4f644a9

pytorchbot added the open source label Oct 27, 2020

neginraoof added 4 commits October 27, 2020 09:32

Update test.sh

9343ffc

Set environ var

f14615a

Merge branch 'neraoof/opset13' of github.com:neginraoof/pytorch into …

19eb56d

…neraoof/opset13

Update test.sh

61855a5

neginraoof changed the title ~~Enable opset 13 ORT tests~~ [ONNX][WIP] Enable opset 13 ORT tests Oct 28, 2020

facebook-github-bot added the cla signed label Oct 30, 2020

neginraoof added 7 commits November 12, 2020 08:18

Enabling more ops for opset 13

be7c21a

change master to main

4e8ef96

Update symbolic_opset13.py

4ab292a

Merge branch 'master' of https://github.com/pytorch/pytorch into nera…

e0ce9bc

…oof/opset13 # Conflicts: # .jenkins/caffe2/test.sh

Merge branch 'master' of https://github.com/pytorch/pytorch into nera…

53b8825

…oof/opset13

Merge branch 'master' of https://github.com/pytorch/pytorch into nera…

0fd0326

…oof/opset13

Flake 8 fix

04f5505

neginraoof marked this pull request as ready for review December 2, 2020 18:15

neginraoof changed the title ~~[ONNX][WIP] Enable opset 13 ORT tests~~ [ONNX] Enable opset 13 ORT tests Dec 2, 2020

heitorschueroff added the module: onnx Related to torch.onnx label Dec 3, 2020

heitorschueroff requested review from BowenBao and lara-hdr December 3, 2020 15:33

heitorschueroff added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Dec 3, 2020

BowenBao reviewed Dec 3, 2020

View reviewed changes

BowenBao and others added 2 commits December 17, 2020 09:59

[ONNX] Support onnx if/loop sequence output in opset 13 - (pytorch#49270

170908b

)

Symbolic function for torch.square (pytorch#49446)

203e181

neginraoof mentioned this pull request Dec 18, 2020

[ONNX] Enable opset 13 ops #49612

Merged

neginraoof force-pushed the neraoof/opset13 branch from 03f815f to 04f5505 Compare December 18, 2020 20:04

neginraoof added 2 commits December 18, 2020 12:18

Clean up tests

f3221f9

Merge branch 'onnx_ms_1' into neraoof/opset13

83272f8

neginraoof closed this Dec 18, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ONNX] Enable opset 13 ORT tests#46903

[ONNX] Enable opset 13 ORT tests#46903
neginraoof wants to merge 16 commits intopytorch:masterfrom
neginraoof:neraoof/opset13

neginraoof commented Oct 27, 2020 •

edited

Loading

Uh oh!

dr-ci bot commented Oct 27, 2020 •

edited

Loading

Uh oh!

BowenBao left a comment

Uh oh!

Uh oh!

BowenBao Dec 3, 2020

Uh oh!

neginraoof Dec 17, 2020

Uh oh!

BowenBao Dec 3, 2020

Uh oh!

BowenBao Dec 3, 2020

Uh oh!

BowenBao Dec 3, 2020

Uh oh!

neginraoof Dec 17, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

neginraoof commented Oct 27, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dr-ci bot commented Oct 27, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CI failures summary and remediations

🕵️ 3 new failures recognized by patterns

pytorch_linux_xenial_cuda9_2_cudnn7_py3_gcc5_4_build (1/3)

pytorch_linux_xenial_py3_6_gcc5_4_build (2/3)

pytorch_linux_xenial_py3_clang7_onnx_ort_test2 (3/3)

🚧 1 fixed upstream failure:

Extra GitHub checks: 4 failed

Uh oh!

BowenBao left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

BowenBao Dec 3, 2020

Choose a reason for hiding this comment

Uh oh!

neginraoof Dec 17, 2020

Choose a reason for hiding this comment

Uh oh!

BowenBao Dec 3, 2020

Choose a reason for hiding this comment

Uh oh!

BowenBao Dec 3, 2020

Choose a reason for hiding this comment

Uh oh!

BowenBao Dec 3, 2020

Choose a reason for hiding this comment

Uh oh!

neginraoof Dec 17, 2020

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

neginraoof commented Oct 27, 2020 •

edited

Loading

dr-ci bot commented Oct 27, 2020 •

edited

Loading