Exporting a model to ONNX; onnx.optimizer #3488
Description
Exporting a model to ONNX leads to an error due to onnx.optimizer being missing, as these packages have been removed from onnx >= 1.9 (exists in onnx<=1.8.1) and moved to onnxoptimizer.
Running with onnx==1.8.1 does not solve the issue. It throws an IndexError: Input is undefined, but I have managed to solve it using the latest onnx and onnxoptimizer.
Instructions To Reproduce the Issue:
Using detectron 0.5 and torch 1.9.0.
Example ONNX model export:
#wget http://images.cocodataset.org/val2017/000000439715.jpg -q -O input.jpg
import onnx
import cv2
import torch
from detectron2 import model_zoo
from detectron2.config import get_cfg
from detectron2.export import export_onnx_model
from detectron2.modeling import build_model
from detectron2.checkpoint import DetectionCheckpointer
import detectron2.data.transforms as T
im = cv2.imread("./input.jpg")
cfg = get_cfg()
# add project-specific config
cfg.merge_from_file(model_zoo.get_config_file("COCO-PanopticSegmentation/panoptic_fpn_R_101_3x.yaml"))
# Find a model from detectron2's model zoo.
cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url("COCO-PanopticSegmentation/panoptic_fpn_R_101_3x.yaml")
cfg.MODEL.DEVICE='cpu'
# Build model and prepare input
model = build_model(cfg)
model.eval()
checkpointer = DetectionCheckpointer(model)
checkpointer.load(cfg.MODEL.WEIGHTS)
aug = T.ResizeShortestEdge([cfg.INPUT.MIN_SIZE_TEST, cfg.INPUT.MIN_SIZE_TEST],
cfg.INPUT.MAX_SIZE_TEST)
height, width = im.shape[:2]
image = aug.get_transform(im).apply_image(im)
image = torch.as_tensor(image.astype("float32").transpose(2, 0, 1))
inputs = {"image": image, "height": height, "width": width}
# Export to Onnx model
onnxModel = export_onnx_model(cfg, model, [inputs])
onnx.save(onnxModel, "test.onnx")
For me, this fails in the onnx.optimizer package with an IndexError, when it appears it should work correctly.
By installing the latest onnx (1.10.1) and onnxoptimizer (0.2.6) packages, I solved it as follows:
-
First,
detectron2/export/caffe2_export.pyfails withNo module named 'onnx.optimizer', so I edit to instead importonnxoptimizer, and similarly replace in lines 68 and 71. -
Next we have
caffe2/python/onnx/backend.pyalso missing onnx.optimizer, so replace import and usage there as well. (I know this is part of the pytorch library).
Finally, the above code now runs successfully and exports the model as expected.
Environment:
"collect_env.py" [1]
---------------------- ---------------------------------------------------------------------------------------
sys.platform linux
Python 3.9.6 (default, Aug 18 2021, 19:38:01) [GCC 7.5.0]
numpy 1.20.3
detectron2 0.5
Compiler GCC 9.3
CUDA compiler CUDA 11.2
detectron2 arch flags 8.6
DETECTRON2_ENV_MODULE <not set>
PyTorch 1.9.0
PyTorch debug build False
GPU available Yes
GPU 0 NVIDIA GeForce RTX 3090 (arch=8.6)
Driver version 465.19.01
CUDA_HOME /usr/local/cuda
Pillow 8.3.1
torchvision 0.10.0
torchvision arch flags 3.5, 5.0, 6.0, 7.0, 7.5, 8.0, 8.6
fvcore 0.1.5.post20210825
iopath 0.1.9
cv2 4.5.3
---------------------- ---------------------------------------------------------------------------------------
PyTorch built with:
- GCC 7.3
- C++ Version: 201402
- Intel(R) oneAPI Math Kernel Library Version 2021.3-Product Build 20210617 for Intel(R) 64 architecture applications
- Intel(R) MKL-DNN v2.1.2 (Git Hash 98be7e8afa711dc9b66c8ff3504129cb82013cdb)
- OpenMP 201511 (a.k.a. OpenMP 4.5)
- NNPACK is enabled
- CPU capability usage: AVX2
- CUDA Runtime 11.1
- NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
- CuDNN 8.0.5
- Magma 2.5.2
- Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.1, CUDNN_VERSION=8.0.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.9.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,
If it's not possible to merge these changed into a pull request, as it is contingent on pytorch as well, then I have posted to help someone with a similar issue.