[ONNX] Update the sequence of initializers in exported graph so that it is as same as inputs. by fatcat-z · Pull Request #49798 · pytorch/pytorch

fatcat-z · 2020-12-23T14:52:27Z

In the process of export, initializers were saved in a C++ map container. Map will automatically sort the elements when an insertion happened. This causes initializers lose the insertion sequence which is supposed to be kept.

So update the backend code to create initializers following up on the inputs sequence in a final stage where initializers were not kept in a map container.

Sync to latest branch.

Get latest changes from pytorch branch.

Get latest changes in master.

Sync from Onnx ms 1

facebook-github-bot · 2020-12-23T15:12:22Z

💊 CI failures summary and remediations

As of commit efd09b9 (more details on the Dr. CI page):

2/2 failures introduced in this PR

🕵️ 2 new failures recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

pytorch_linux_backward_compatibility_check_test (1/2)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Jan 05 09:53:32 The PR is introducing backward incompatible changes to the operator library. Please contact PyTorch team to confirm whether this change is wanted or not.

Jan 05 09:53:32 processing existing schema:  alltoall(__torch__.torch.classes.dist_c10d.ProcessGroup _0, Tensor[] _1, Tensor[] _2) -> (__torch__.torch.classes.dist_c10d.Work _0)
Jan 05 09:53:32 processing existing schema:  send(__torch__.torch.classes.dist_c10d.ProcessGroup _0, Tensor[] _1, int _2, int _3) -> (__torch__.torch.classes.dist_c10d.Work _0)
Jan 05 09:53:32 processing existing schema:  recv(__torch__.torch.classes.dist_c10d.ProcessGroup _0, Tensor[] _1, int _2, int _3) -> (__torch__.torch.classes.dist_c10d.Work _0)
Jan 05 09:53:32 processing existing schema:  recv_anysource(__torch__.torch.classes.dist_c10d.ProcessGroup _0, Tensor[] _1, int _2) -> (__torch__.torch.classes.dist_c10d.Work _0)
Jan 05 09:53:32 processing existing schema:  barrier(__torch__.torch.classes.dist_c10d.ProcessGroup _0) -> (__torch__.torch.classes.dist_c10d.Work _0)
Jan 05 09:53:32 processing existing schema:  __init__(__torch__.torch.classes.dist_c10d.frontend _0) -> (None _0)
Jan 05 09:53:32 processing existing schema:  new_process_group_helper(__torch__.torch.classes.dist_c10d.frontend _0, int _1, int _2, int[] _3, str _4, __torch__.torch.classes.dist_c10d.Store _5, str? _6, int _7) -> (__torch__.torch.classes.dist_c10d.ProcessGroup _0)
Jan 05 09:53:32 processing existing schema:  get_process_group_by_name(__torch__.torch.classes.dist_c10d.frontend _0, str _1) -> (__torch__.torch.classes.dist_c10d.ProcessGroup _0)
Jan 05 09:53:32 processing existing schema:  get_name_of_process_group(__torch__.torch.classes.dist_c10d.frontend _0, __torch__.torch.classes.dist_c10d.ProcessGroup _1) -> (str _0)
Jan 05 09:53:32 processing existing schema:  __init__(__torch__.torch.classes.dist_rpc.WorkerInfo _0, str _1, int _2) -> (None _0)
Jan 05 09:53:32 The PR is introducing backward incompatible changes to the operator library. Please contact PyTorch team to confirm whether this change is wanted or not. 
Jan 05 09:53:32 
Jan 05 09:53:32 Broken ops: [
Jan 05 09:53:32 	aten::_test_ambiguous_defaults.a(Tensor dummy, int a=1, int b=1) -> (Tensor)
Jan 05 09:53:32 	aten::_test_ambiguous_defaults.b(Tensor dummy, int a=2, str b="2") -> (Tensor)
Jan 05 09:53:32 ]
Jan 05 09:53:32 =================== sccache compilation log ===================
Jan 05 09:53:32 + cleanup
Jan 05 09:53:32 + retcode=1
Jan 05 09:53:32 + set +x
Jan 05 09:53:32 =========== If your build fails, please take a look at the log above for possible reasons ===========

pytorch_windows_vs2019_py36_cuda10.1_test1 (2/2)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

RuntimeError: test_nn failed!

  test_ReflectionPad2d (__main__.TestNN) ... ok (1.176s)
  test_ReflectionPad2d_alert_nondeterministic_cuda (__main__.TestNN) ... ok (0.011s)
  test_ReflectionPad2d_cuda (__main__.TestNN) ... ok (0.014s)
  test_ReplicationPad1d (__main__.TestNN) ... ok (0.069s)
  test_ReplicationPad1d_alert_nondeterministic_cuda (__main__.TestNN) ... ok (0.014s)
  test_ReplicationPad1d_cuda (__main__.TestNN) ... Traceback (most recent call last):
  File "run_test.py", line 910, in <module>
    main()
  File "run_test.py", line 889, in main
    raise RuntimeError(err_message)
RuntimeError: test_nn failed!

(base) circleci@PACKER-5FD865C5 C:\Users\circleci\project\test>if ERRORLEVEL 1 exit /b 1 
+ cleanup
+ retcode=1
+ set +x


Exited with code exit status 1

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

This comment has been revised 21 times.

torch/csrc/jit/serialization/export.cpp

BowenBao · 2021-01-05T05:10:43Z

Thanks @fatcat-z, could you check the failed onnx_ort_test1/onnx_ort_test2 CI tests and fix if related?

test/onnx/test_pytorch_onnx_onnxruntime.py

fatcat-z · 2021-01-05T11:17:14Z

Thanks @fatcat-z, could you check the failed onnx_ort_test1/onnx_ort_test2 CI tests and fix if related?

It has been fixed by updating expect file.

…it is as same as inputs. (pytorch#49798) * [ONNX] Support onnx if/loop sequence output in opset 13 - (pytorch#49270) * Symbolic function for torch.square (pytorch#49446) * [ONNX] Support onnx if/loop sequence output in opset 13 - (pytorch#49270) * Symbolic function for torch.square (pytorch#49446) * Update code so that initializers' sequence is as same as inputs. * Correct the format according to flake8. * Correct the format by clang-format. * Add a new test for script model. * Update expect files for Test_Operators tests. Co-authored-by: Bowen Bao <[email protected]> Co-authored-by: David Fan <[email protected]>

Summary: [ONNX] ONNX dev branch merge 01-06-2021 - [ONNX] Support onnx if/loop sequence output in opset 13 - (#49270) - Symbolic function for torch.square (#49446) - [ONNX] Add checks in ONNXSetDynamicInputShape (#49783) … - [ONNX] Enable export af aten::__derive_index (#49514) … - [ONNX] Update symbolic for unfold (#49378) … - [ONNX] Update the sequence of initializers in exported graph so that it is as same as inputs. (#49798) - [ONNX] Enable opset 13 ops (#49612) … - [ONNX] Improve error message for supported model input types in ONNX export API. (#50119) - [ONNX] Add a post-pass for If folding (#49410) Pull Request resolved: #50163 Reviewed By: pbelevich Differential Revision: D25821059 Pulled By: SplitInfinity fbshipit-source-id: 9f511a93d9d5812d0ab0a49d61ed0fa5f8066948

fatcat-z and others added 10 commits December 4, 2020 14:34

Merge pull request #1 from pytorch/master

760fc5d

Sync to latest branch.

Merge pull request #2 from pytorch/master

4520ea7

Get latest changes from pytorch branch.

Merge pull request #3 from pytorch/master

c46c37f

Get latest changes in master.

[ONNX] Support onnx if/loop sequence output in opset 13 - (#49270)

170908b

Symbolic function for torch.square (#49446)

203e181

Merge pull request #4 from pytorch/onnx_ms_1

bb59c3d

Sync from Onnx ms 1

[ONNX] Support onnx if/loop sequence output in opset 13 - (#49270)

29dd23f

Symbolic function for torch.square (#49446)

f7c63eb

Merge pull request #5 from pytorch/onnx_ms_1

46d103e

Sync from Onnx ms 1

Update code so that initializers' sequence is as same as inputs.

9ce4b6b

facebook-github-bot added cla signed oncall: jit Add this issue/PR to JIT oncall triage queue labels Dec 23, 2020

pytorchbot added the open source label Dec 23, 2020

fatcat-z added 2 commits December 23, 2020 23:23

Correct the format according to flake8.

0851a82

Correct the format by clang-format.

85a871a

neginraoof reviewed Dec 23, 2020

View reviewed changes

torch/csrc/jit/serialization/export.cpp Show resolved Hide resolved

Add a new test for script model.

defd9b0

BowenBao force-pushed the onnx_ms_1 branch from ab060a1 to 1baebbb Compare January 4, 2021 18:29

Merge branch 'onnx_ms_1' into master

0e4e1c5

BowenBao reviewed Jan 5, 2021

View reviewed changes

test/onnx/test_pytorch_onnx_onnxruntime.py Outdated Show resolved Hide resolved

Update expect files for Test_Operators tests.

efd09b9

BowenBao approved these changes Jan 5, 2021

View reviewed changes

BowenBao merged commit 97a8af1 into pytorch:onnx_ms_1 Jan 5, 2021

spandantiwari mentioned this pull request Jan 7, 2021

[ONNX] ONNX dev branch merge 01-06-2021 #50163

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ONNX] Update the sequence of initializers in exported graph so that it is as same as inputs.#49798

[ONNX] Update the sequence of initializers in exported graph so that it is as same as inputs.#49798
BowenBao merged 15 commits intopytorch:onnx_ms_1from
fatcat-z:master

fatcat-z commented Dec 23, 2020

Uh oh!

facebook-github-bot commented Dec 23, 2020 •

edited

Loading

Uh oh!

Uh oh!

BowenBao commented Jan 5, 2021

Uh oh!

Uh oh!

fatcat-z commented Jan 5, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

fatcat-z commented Dec 23, 2020

Uh oh!

facebook-github-bot commented Dec 23, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CI failures summary and remediations

🕵️ 2 new failures recognized by patterns

pytorch_linux_backward_compatibility_check_test (1/2)

pytorch_windows_vs2019_py36_cuda10.1_test1 (2/2)

Uh oh!

Uh oh!

BowenBao commented Jan 5, 2021

Uh oh!

Uh oh!

fatcat-z commented Jan 5, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

facebook-github-bot commented Dec 23, 2020 •

edited

Loading