[torchlib] Use Optional in type annotations for linear by justinchuby · Pull Request #2024 · microsoft/onnxscript

justinchuby · 2025-01-20T15:18:48Z

Fix #2023

codecov · 2025-01-20T15:36:29Z

❌ 53 Tests Failed:

Tests completed	Failed	Passed	Skipped
11815	53	11762	2454

View the top 3 failed tests by shortest run time

onnxscript.backend.onnx_export_test.TestOnnxBackEnd::test_export2python_produces_correct_onnx_script_model_0417_test_gemm_default_zero_bias

Stack Traces | 0.003s run time

onnxscript\backend\onnx_export_test.py:137: in extract_functions
    mod = importlib.import_module(import_name)
C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
E   ModuleNotFoundError: No module named 'tests.onnx_backend_test_code.test_gemm_default_zero_bias'

The above exception was the direct cause of the following exception:
.nox\test_onnx_weekly\Lib\site-packages\parameterized\parameterized.py:620: in standalone_func
    return func(*(a + p.args), **p.kwargs, **kw)
onnxscript\backend\onnx_export_test.py:271: in test_export2python_produces_correct_onnx_script_model
    functions = extract_functions(backend_test.name, code, self.test_folder)
onnxscript\backend\onnx_export_test.py:139: in extract_functions
    raise AssertionError(
E   AssertionError: Unable to import 'tests.onnx_backend_test_code.test_gemm_default_zero_bias' (e=No module named 'tests.onnx_backend_test_code.test_gemm_default_zero_bias') (file: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_gemm_default_zero_bias.py', absolute path: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_gemm_default_zero_bias.py', current folder: D:\a\onnxscript\onnxscript
E   ---- CONTENT --
E   import numpy
E   from onnx import TensorProto
E   from onnx.helper import make_tensor
E   from onnxscript import script, external_tensor
E   from onnxscript.values import Opset
E   from onnxscript.onnx_types import FLOAT
E   from onnxscript.onnx_opset import opset13
E   
E   @script()
E   def bck_test_gemm_default_zero_bias(a: FLOAT[3,5], b: FLOAT[5,4], c: FLOAT[1,4]) -> (FLOAT[3,4]):
E       y = opset13.Gemm(a, b, c)
E       return y

onnxscript.backend.onnx_export_test.TestOnnxBackEnd::test_export2python_produces_correct_onnx_script_model_0470_test_hardmax_one_hot

Stack Traces | 0.003s run time

onnxscript\backend\onnx_export_test.py:137: in extract_functions
    mod = importlib.import_module(import_name)
C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
E   ModuleNotFoundError: No module named 'tests.onnx_backend_test_code.test_hardmax_one_hot'

The above exception was the direct cause of the following exception:
.nox\test_torch_nightly\Lib\site-packages\parameterized\parameterized.py:620: in standalone_func
    return func(*(a + p.args), **p.kwargs, **kw)
onnxscript\backend\onnx_export_test.py:271: in test_export2python_produces_correct_onnx_script_model
    functions = extract_functions(backend_test.name, code, self.test_folder)
onnxscript\backend\onnx_export_test.py:139: in extract_functions
    raise AssertionError(
E   AssertionError: Unable to import 'tests.onnx_backend_test_code.test_hardmax_one_hot' (e=No module named 'tests.onnx_backend_test_code.test_hardmax_one_hot') (file: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_hardmax_one_hot.py', absolute path: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_hardmax_one_hot.py', current folder: D:\a\onnxscript\onnxscript
E   ---- CONTENT --
E   import numpy
E   from onnx import TensorProto
E   from onnx.helper import make_tensor
E   from onnxscript import script, external_tensor
E   from onnxscript.values import Opset
E   from onnxscript.onnx_types import FLOAT
E   from onnxscript.onnx_opset import opset13
E   
E   @script()
E   def bck_test_hardmax_one_hot(x: FLOAT[1,4]) -> (FLOAT[1,4]):
E       y = opset13.Hardmax(x)
E       return y

onnxscript.backend.onnx_export_test.TestOnnxBackEnd::test_export2python_produces_correct_onnx_script_model_0908_test_reduce_prod_negative_axes_keepdims_example

Stack Traces | 0.003s run time

onnxscript\backend\onnx_export_test.py:137: in extract_functions
    mod = importlib.import_module(import_name)
C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
E   ModuleNotFoundError: No module named 'tests.onnx_backend_test_code.test_reduce_prod_negative_axes_keepdims_example'

The above exception was the direct cause of the following exception:
.nox\test_onnx_weekly\Lib\site-packages\parameterized\parameterized.py:620: in standalone_func
    return func(*(a + p.args), **p.kwargs, **kw)
onnxscript\backend\onnx_export_test.py:271: in test_export2python_produces_correct_onnx_script_model
    functions = extract_functions(backend_test.name, code, self.test_folder)
onnxscript\backend\onnx_export_test.py:139: in extract_functions
    raise AssertionError(
E   AssertionError: Unable to import 'tests.onnx_backend_test_code.test_reduce_prod_negative_axes_keepdims_example' (e=No module named 'tests.onnx_backend_test_code.test_reduce_prod_negative_axes_keepdims_example') (file: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_reduce_prod_negative_axes_keepdims_example.py', absolute path: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_reduce_prod_negative_axes_keepdims_example.py', current folder: D:\a\onnxscript\onnxscript
E   ---- CONTENT --
E   import numpy
E   from onnx import TensorProto
E   from onnx.helper import make_tensor
E   from onnxscript import script, external_tensor
E   from onnxscript.values import Opset
E   from onnxscript.onnx_types import FLOAT, INT64
E   from onnxscript.onnx_opset import opset18
E   
E   @script()
E   def bck_test_reduce_prod_negative_axes_keepdims_example(data: FLOAT[3,2,2], axes: INT64[1]) -> (FLOAT[3,1,2]):
E       reduced = opset18.ReduceProd(data, axes, keepdims=1)
E       return reduced

To view more test analytics, go to the Test Analytics Dashboard
📢 Thoughts on this report? Let us know!

kevalmorabia97 · 2025-01-21T15:45:41Z

This repository does use | operator instead of Union or Optional for type annotations quite a lot in most files which is not supported for Python<3.10. Does that mean onnxscript should not be used for Python<3.10?

gramalingam · 2025-01-22T20:34:23Z

This repository does use | operator instead of Union or Optional for type annotations quite a lot in most files which is not supported for Python<3.10. Does that mean onnxscript should not be used for Python<3.10?

I think the issue is only when it is used with TypeVars? Other uses are fine because of from __future__ import annotations, I think?

justinchuby · 2025-01-22T20:41:34Z

This is an issue when we want to evaluate the function signature to get the type. This is why I think it maybe better to decorate the function with type information explicitly to avoid having to parse python types and bridge different python versions (#2011)

justinchuby · 2025-01-22T20:42:36Z

And yes as long as we don't evaluate the annotations, __future__ import annotations turns all annotations into strings so nothing hurts.

### Description This PR adds fusions for [Google's SigLIP model](https://huggingface.co/google/siglip-base-patch16-224/) and Microsoft's internal conformer-encoder model. Here is an example of how to run the ORT transformer optimizer for the SigLIP model. ``` $ git clone https://github.com/microsoft/onnxruntime $ cd onnxruntime/onnxruntime/python/tools/transformers $ python3 optimizer.py --input /path/to/model.onnx --output /path/to/model_opt.onnx --model_type clip --num_heads 16 --hidden_size 1152 --use_external_data_format --opt_level 0 --disable_shape_inference ``` Here is an example of how to run the ORT transformer optimizer for the conformer-encoder model. ``` $ git clone https://github.com/microsoft/onnxruntime $ cd onnxruntime/onnxruntime/python/tools/transformers $ python3 optimizer.py --input /path/to/model.onnx --output /path/to/model_opt.onnx --model_type conformer --num_heads 16 --hidden_size 1024 --use_external_data_format --opt_level 0 --disable_shape_inference --convert_attribute ``` ### Motivation and Context This PR helps optimize multi-modal models that use SigLIP for the vision encoder and conformer-encoder for the speech encoder. This PR uses changes from the following PRs: - pytorch/pytorch#144801 - microsoft/onnxscript#2018 - microsoft/onnxscript#2019 - microsoft/onnxscript#2020 - microsoft/onnxscript#2021 - microsoft/onnxscript#2022 - microsoft/onnxscript#2024 - microsoft/onnxscript#2025 - microsoft/onnxscript#2029 - microsoft/onnxscript#2033 ### Introduction of ONNX Script This PR introduces [ONNX Script](https://github.com/microsoft/onnxscript) into the ORT transformer optimizer as an optional step via the `fold_transpose_initializers()` method of the `DynamoOnnxHelper` class.

[torchlib] Use Optional in type annotations for linear

8a454bc

justinchuby requested a review from titaiwangms January 20, 2025 15:18

justinchuby enabled auto-merge (squash) January 20, 2025 15:19

titaiwangms approved these changes Jan 21, 2025

View reviewed changes

justinchuby merged commit 7582138 into main Jan 21, 2025
21 of 27 checks passed

justinchuby deleted the justinchuby-patch-1 branch January 21, 2025 16:17

kunal-vaishnavi mentioned this pull request Jan 29, 2025

Add fusions for SigLIP and Conformer-Encoder microsoft/onnxruntime#23528

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[torchlib] Use Optional in type annotations for linear#2024

[torchlib] Use Optional in type annotations for linear#2024
justinchuby merged 1 commit intomainfrom
justinchuby-patch-1

justinchuby commented Jan 20, 2025 •

edited

Loading

Uh oh!

codecov bot commented Jan 20, 2025 •

edited

Loading

Uh oh!

kevalmorabia97 commented Jan 21, 2025 •

edited

Loading

Uh oh!

Uh oh!

gramalingam commented Jan 22, 2025

Uh oh!

justinchuby commented Jan 22, 2025

Uh oh!

justinchuby commented Jan 22, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

justinchuby commented Jan 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Jan 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ 53 Tests Failed:

Uh oh!

kevalmorabia97 commented Jan 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

gramalingam commented Jan 22, 2025

Uh oh!

justinchuby commented Jan 22, 2025

Uh oh!

justinchuby commented Jan 22, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

justinchuby commented Jan 20, 2025 •

edited

Loading

codecov bot commented Jan 20, 2025 •

edited

Loading

kevalmorabia97 commented Jan 21, 2025 •

edited

Loading