Add op (AvgPool) | feat (torchlib) by titaiwangms · Pull Request #754 · microsoft/onnxscript

titaiwangms · 2023-05-31T17:41:48Z

1D and 3D

Fix can save the op after opset 19, but not before it. So an xfail is created when ceil_mode=True

titaiwangms · 2023-05-31T23:03:44Z

I am blocked by ceil_mode. It seems that the ceil_mode in ORT is not following ONNX spec:

import numpy as np
from onnxscript.function_libs.torch_lib.ops.nn import aten_avg_pool2d

x = np.array(
    [
        [
            [
                [1, 2, 3, 4],
                [5, 6, 7, 8],
                [9, 10, 11, 12],
                [13, 14, 15, 16],
            ]
        ]
    ]
).astype(np.float32)
kernal = [3, 3]
stride = [2, 2]

print(aten_avg_pool2d(x, kernel_size=kernal, stride=stride, ceil_mode=True))
# Tensor(array([[[[6., 5.], [8., 6.]]]], dtype=float32))

# https://github.com/onnx/onnx/blob/main/docs/Operators.md#averagepool
# The answer should be:
# np.array([[[[6, 7.5], [12, 13.5]]]]).astype(np.float32)

So the difference is that in ceil_mode, ORT tends to divide the value by kernal_size no matter the right padding exists or not.
But I don't think they would take this as repro..

titaiwangms · 2023-06-01T00:13:50Z

I am blocked by ceil_mode. It seems that the ceil_mode in ORT is not following ONNX spec:

import numpy as np
from onnxscript.function_libs.torch_lib.ops.nn import aten_avg_pool2d

x = np.array(
    [
        [
            [
                [1, 2, 3, 4],
                [5, 6, 7, 8],
                [9, 10, 11, 12],
                [13, 14, 15, 16],
            ]
        ]
    ]
).astype(np.float32)
kernal = [3, 3]
stride = [2, 2]

print(aten_avg_pool2d(x, kernel_size=kernal, stride=stride, ceil_mode=True))
# Tensor(array([[[[6., 5.], [8., 6.]]]], dtype=float32))

# https://github.com/onnx/onnx/blob/main/docs/Operators.md#averagepool
# The answer should be:
# np.array([[[[6, 7.5], [12, 13.5]]]]).astype(np.float32)

So the difference is that in ceil_mode, ORT tends to divide the value by kernal_size no matter the right padding exists or not. But I don't think they would take this as repro..

Just found this issue is caused by "count_include_pad=True will pad the right side even if it's not padded". So the following code would be able to solve this:

x = np.array(
    [
        [
            [
                [1, 2, 3, 4],
                [5, 6, 7, 8],
                [9, 10, 11, 12],
                [13, 14, 15, 16],
            ]
        ]
    ]
).astype(np.float32)

import onnxscript
from onnxscript.onnx_opset import opset18 as op

@onnxscript.script(default_opset=op)
def avg_pool(x):
    result = op.AveragePool(x, kernel_shape=[3,3], strides=[2,2], ceil_mode=True, count_include_pad=False)
    return result

print(avg_pool(x))

Although I still think this doesn't make sense.

titaiwangms · 2023-06-01T00:16:56Z

However, the above doesn't solve the root cause which is that in ORT, the last slide of window tends to dividing the value with kernal size regardless the remain pixel+pads seems like a bug.

titaiwangms · 2023-06-01T01:10:14Z

Not sure if it's a spec issue or implementation: onnx/onnx#5276

onnxscript/function_libs/torch_lib/ops/nn.py

onnxscript/tests/function_libs/torch_lib/ops_test_data.py

titaiwangms · 2023-06-02T00:54:40Z

microsoft/onnxruntime#16203

codecov · 2023-07-13T16:10:16Z

Codecov Report

Merging #754 (5b5b715) into main (65d7c03) will decrease coverage by 0.04%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #754      +/-   ##
==========================================
- Coverage   76.67%   76.64%   -0.04%     
==========================================
  Files         112      112              
  Lines       13496    13508      +12     
  Branches     1365     1366       +1     
==========================================
+ Hits        10348    10353       +5     
- Misses       2809     2817       +8     
+ Partials      339      338       -1

Impacted Files	Coverage Δ
onnxscript/function_libs/torch_lib/ops/core.py	`77.10% <ø> (+0.02%)`	⬆️
onnxscript/function_libs/torch_lib/ops/nn.py	`71.42% <100.00%> (+1.08%)`	⬆️
...ipt/tests/function_libs/torch_lib/ops_test_data.py	`96.78% <100.00%> (+0.01%)`	⬆️

... and 1 file with indirect coverage changes

onnxscript/tests/function_libs/torch_lib/ops_test_data.py

… shapes" In #87892, to pick up the corner cases found in #71549, the PR falls back the implementation of AvgPool to the way opset 9 implementing. However, it introduces a regression on dynamic shape cases found in #101397. This PR refactors the AvgPool op with the same implementation we have in onnxscript: microsoft/onnxscript#754. However, the corner case with `ceil_mode` remains unsolved in onnxruntime: microsoft/onnxruntime#16203. The calculuation on the last value of each dimension is different between ORT and PyTorch. But the fix can be proved in: microsoft/onnxruntime#16752, and it supports AvgPool since opset19. [ghstack-poisoned]

In #87892, to pick up the corner cases found in #71549, the PR falls back the implementation of AvgPool to the way opset 9 implementing. However, it introduces a regression on dynamic shape cases found in #101397. This PR refactors the AvgPool op with the same implementation we have in onnxscript: microsoft/onnxscript#754. However, the corner case with `ceil_mode` remains unsolved in onnxruntime: microsoft/onnxruntime#16203. The calculuation on the last value of each dimension is different between ORT and PyTorch. But the fix can be proved in: microsoft/onnxruntime#16752, and it supports AvgPool since opset19. [ghstack-poisoned]

… shapes" In #87892, to pick up the corner cases found in #71549, the PR falls back the implementation of AvgPool to the way opset 9 implementing. However, it introduces a regression on dynamic shape cases found in #101397. This PR refactors the AvgPool op with the same implementation we have in onnxscript: microsoft/onnxscript#754. However, the corner case with `ceil_mode` remains unsolved in onnxruntime: microsoft/onnxruntime#16203. The calculuation on the last value of each dimension is different between ORT and PyTorch. But the fix can be proved in: microsoft/onnxruntime#16752, and it supports AvgPool since opset19. [ghstack-poisoned]

In #87892, to pick up the corner cases found in #71549, the PR falls back the implementation of AvgPool to the way opset 9 implementing. However, it introduces a regression on dynamic shape cases found in #101397. This PR refactors the AvgPool op with the same implementation we have in onnxscript: microsoft/onnxscript#754. However, the corner case with `ceil_mode` remains unsolved in onnxruntime: microsoft/onnxruntime#16203. The calculuation on the last value of each dimension is different between ORT and PyTorch. But the fix can be proved in: microsoft/onnxruntime#16752, and it supports AvgPool since opset19. [ghstack-poisoned]

… shapes" In #87892, to pick up the corner cases found in #71549, the PR falls back the implementation of AvgPool to the way opset 9 implementing. However, it introduces a regression on dynamic shape cases found in #101397. This PR refactors the AvgPool op with the same implementation we have in onnxscript: microsoft/onnxscript#754. However, the corner case with `ceil_mode` remains unsolved in onnxruntime: microsoft/onnxruntime#16203. The calculuation on the last value of each dimension is different between ORT and PyTorch. But the fix can be proved in: microsoft/onnxruntime#16752, and it supports AvgPool since opset19. [ghstack-poisoned]

In #87892, to pick up the corner cases found in #71549, the PR falls back the implementation of AvgPool to the way opset 9 implementing. However, it introduces a regression on dynamic shape cases found in #101397. This PR refactors the AvgPool op with the same implementation we have in onnxscript: microsoft/onnxscript#754. However, the corner case with `ceil_mode` remains unsolved in onnxruntime: microsoft/onnxruntime#16203. The calculuation on the last value of each dimension is different between ORT and PyTorch. But the fix can be proved in: microsoft/onnxruntime#16752, and it supports AvgPool since opset19. [ghstack-poisoned]

… shapes" In #87892, to pick up the corner cases found in #71549, the PR falls back the implementation of AvgPool to the way opset 9 implementing. However, it introduces a regression on dynamic shape cases found in #101397. This PR refactors the AvgPool op with the same implementation we have in onnxscript: microsoft/onnxscript#754. However, the corner case with `ceil_mode` remains unsolved in onnxruntime: microsoft/onnxruntime#16203. The calculuation on the last value of each dimension is different between ORT and PyTorch. But the fix can be proved in: microsoft/onnxruntime#16752, and it supports AvgPool since opset19. [ghstack-poisoned]

In #87892, to pick up the corner cases found in #71549, the PR falls back the implementation of AvgPool to the way opset 9 implementing. However, it introduces a regression on dynamic shape cases found in #101397. This PR refactors the AvgPool op with the same implementation we have in onnxscript: microsoft/onnxscript#754. However, the corner case with `ceil_mode` remains unsolved in onnxruntime: microsoft/onnxruntime#16203. The calculuation on the last value of each dimension is different between ORT and PyTorch. But the fix can be proved in: microsoft/onnxruntime#16752, and it supports AvgPool since opset19. [ghstack-poisoned]

In #87892, to pick up the corner cases found in #71549, the PR falls back the implementation of AvgPool to the way opset 9 implementing. However, it introduces a regression on dynamic shape cases found in #101397. This PR refactors the AvgPool op with the same implementation we have in onnxscript: microsoft/onnxscript#754. However, the corner case with `count_include_pad` remains unsolved in onnxruntime: microsoft/onnxruntime#16203. The calculuation on the last value of each dimension is different between ORT and PyTorch. But the fix can be proved in: microsoft/onnxruntime#16752, and it supports AvgPool since opset19. Pull Request resolved: #105683 Approved by: https://github.com/thiagocrepaldi

titaiwangms added 3 commits May 30, 2023 23:46

add draft

fa3290d

Use avg_pool2d format

251489d

Merge branch 'main' into titaiwang/add_avg_pool_fami

9776ced

titaiwangms added the module: torchlib Related to the torch/aten function lib in development label May 31, 2023

adjust padding

f98aaaa

titaiwangms requested review from fatcat-z, justinchuby and xiaowuhu May 31, 2023 23:00

titaiwangms marked this pull request as ready for review May 31, 2023 23:00

titaiwangms added the help wanted Extra attention is needed label May 31, 2023

fatcat-z reviewed Jun 1, 2023

View reviewed changes

onnxscript/function_libs/torch_lib/ops/nn.py Outdated Show resolved Hide resolved

fatcat-z reviewed Jun 1, 2023

View reviewed changes

onnxscript/tests/function_libs/torch_lib/ops_test_data.py Outdated Show resolved Hide resolved

titaiwangms added 2 commits June 1, 2023 21:14

address comment

4d8b5b8

Merge branch 'main' into titaiwang/add_avg_pool_fami

bf53cdd

xiaowuhu approved these changes Jun 2, 2023

View reviewed changes

Merge branch 'main' into titaiwang/add_avg_pool_fami

60b6030

justinchuby and others added 4 commits June 15, 2023 10:43

Merge branch 'main' into titaiwang/add_avg_pool_fami

243e686

Merge branch 'main' into titaiwang/add_avg_pool_fami

b5ebe92

merge main

1c3ac2e

fix merged conflict

d6bf99b

titaiwangms and others added 4 commits July 18, 2023 16:13

merge main

e39b167

Merge branch 'main' into titaiwang/add_avg_pool_fami

dfd57e4

add tests

4e698ca

add xfail on ceil_mode

769675b

justinchuby reviewed Jul 18, 2023

View reviewed changes

onnxscript/tests/function_libs/torch_lib/ops_test_data.py Outdated Show resolved Hide resolved

titaiwangms removed the help wanted Extra attention is needed label Jul 18, 2023

titaiwangms and others added 3 commits July 18, 2023 23:04

update and have more specific xfail

a425eb9

Merge branch 'main' into titaiwang/add_avg_pool_fami

227c77d

Merge branch 'main' into titaiwang/add_avg_pool_fami

5b5b715

justinchuby approved these changes Jul 19, 2023

View reviewed changes

titaiwangms merged commit 3797447 into microsoft:main Jul 19, 2023

This was referenced Jul 20, 2023

[ONNX] Support of AvgPool2D when ceil_mode is True has disappeared with Torch 2.0 pytorch/pytorch#101397

Closed

[ONNX] Refactor AvgPool to support dynamic shapes pytorch/pytorch#105683

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add op (AvgPool) | feat (torchlib)#754

Add op (AvgPool) | feat (torchlib)#754
titaiwangms merged 18 commits intomicrosoft:mainfrom
titaiwangms:titaiwang/add_avg_pool_fami

titaiwangms commented May 31, 2023 •

edited

Loading

Uh oh!

titaiwangms commented May 31, 2023 •

edited

Loading

Uh oh!

titaiwangms commented Jun 1, 2023

Uh oh!

titaiwangms commented Jun 1, 2023

Uh oh!

titaiwangms commented Jun 1, 2023 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

titaiwangms commented Jun 2, 2023

Uh oh!

codecov bot commented Jul 13, 2023 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

titaiwangms commented May 31, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

titaiwangms commented May 31, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

titaiwangms commented Jun 1, 2023

Uh oh!

titaiwangms commented Jun 1, 2023

Uh oh!

titaiwangms commented Jun 1, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

titaiwangms commented Jun 2, 2023

Uh oh!

codecov bot commented Jul 13, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

titaiwangms commented May 31, 2023 •

edited

Loading

titaiwangms commented May 31, 2023 •

edited

Loading

titaiwangms commented Jun 1, 2023 •

edited

Loading

codecov bot commented Jul 13, 2023 •

edited

Loading