[onnxruntime] Add new port by pash-msft · Pull Request #14903 · microsoft/vcpkg

pash-msft · 2020-12-02T09:59:59Z

Describe the pull request
Add new port for OnnxRuntime:
https://github.com/microsoft/onnxruntime

What does your PR fix? Fixes [New Port Request] onnxruntime #14257
This is a new port
Which triplets are supported/not supported? Have you updated the CI baseline?
Only supported triplets are x86-windows-static and x86-linux-static
Does your PR follow the maintainer guide?
Yes

ghost · 2020-12-02T10:00:13Z

All CLA requirements met.

ports/onnxruntime/portfile.cmake

ports/onnxruntime/vcpkg.json

NancyLi1013 · 2020-12-03T03:18:44Z

@pash-msft
Could you please sign CLA first?

jthelin · 2020-12-03T16:36:26Z

Could you please sign CLA first?

I am working with @pash-msft to get his Github account correctly linked with his FTE microsoft.com account in the internal systems, and make the CLA Bot happy.

Thank you @NancyLi1013 for proceeding with the code review for this change while we work through the processes to get all the correct GH account linkage and GH org membership settings sorted out in the systems.

pash-msft · 2020-12-04T06:45:03Z

Could you please sign CLA first?

I am working with @pash-msft to get his Github account correctly linked with his FTE microsoft.com account in the internal systems, and make the CLA Bot happy.

Thank you @NancyLi1013 for proceeding with the code review for this change while we work through the processes to get all the correct GH account linkage and GH org membership settings sorted out in the systems.

Thanks Jorgen. I followed your suggestion. The PR has now met CLA requirement.

ports/onnxruntime/portfile.cmake

NancyLi1013 · 2020-12-14T06:44:59Z

The failures on x64-linux:

Use gtest from submodule
CMake Error at /mnt/vcpkg-ci/downloads/tools/cmake-3.18.4-linux/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
  Could NOT find PythonInterp: Found unsuitable version "2.7.17", but
  required is at least "3.4" (found /usr/bin/python)
Call Stack (most recent call first):
  /mnt/vcpkg-ci/downloads/tools/cmake-3.18.4-linux/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:456 (_FPHSA_FAILURE_MESSAGE)
  /mnt/vcpkg-ci/downloads/tools/cmake-3.18.4-linux/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPythonInterp.cmake:169 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  /agent/_work/1/s/scripts/buildsystems/vcpkg.cmake:625 (_find_package)
  CMakeLists.txt:506 (find_package)

Could you please look into this and try to fix it?

ports/onnxruntime/portfile.cmake

NancyLi1013 · 2020-12-23T10:27:38Z

scripts/ci.baseline.txt

+onnxruntime:x64-uwp=fail
+onnxruntime:x64-windows=fail
+onnxruntime:x64-windows-static=fail
+onnxruntime:x86-windows=fail


Why do you set all triplets fail here? If not supports, we should add supports field to vckg.json.

I was getting CI errors for the above triplets, the suggested solution was to add expected failures here. If I use vcpkg_fail_port_install, how do I tell CI pipeline to ignore failure on these unsupported ARCH and Target. Thanks for your feedback.

You can add supports": "windows & !x86 & !uwp & !static & !arm & !wasm32 to vcpkg.json and remove these codes in ci.baseline.txt.

Please refer to https://github.com/microsoft/vcpkg/blob/master/ports/onnxruntime-gpu/vcpkg.json#L6.

I am not able to find right values here. There are only two triplets to be supported:
x64-linux
x64-windows-static-md
Could you please suggest me changes for vcpkg.json and portfile.cmake? Looks like I have to keep it the way it is. Let me know. Thanks.

What about x64-windows and x64-windows-static? Are they not supported?

x64-windows-static-md is community triplet, which will not be tested on CI pipeline.

If so, currently this port only supports Linux. We can add supports: linux to vcpkg.json and vcpkg_fail_port_install(ON_TARGET "windows" "osx") to portfile.cmake.

We plan to support x64-windows and x64-windows-static. Currently, we don't use these builds in our application. This can be extended to support future platforms.
Our application will use x64-windows-static-md build configuration. So, we can not have vcpkg_fail_port_install(ON_TARGET "windows" "osx") in the portfile.cmake correct?

If you are OK , can we have it approved. Have addressed other feedbacks. Thank you.

If these triplets are currently not officially supported, we can add supports instead of adding them here.

Yes. We plan to add support incrementally. Thanks.

Please remove these codes here.

NancyLi1013 · 2021-01-08T07:28:25Z

The latest failures on linux:

There should be no empty directories in /mnt/vcpkg-ci/packages/onnxruntime_x64-linux
The following empty directories were found:

    /mnt/vcpkg-ci/packages/onnxruntime_x64-linux/include/onnxruntime/core/providers/rocm/math

If a directory should be populated but is not, this might indicate an error in the portfile.
If the directories are not needed and their creation cannot be disabled, use something like this in the portfile to remove them:

    file(REMOVE_RECURSE "${CURRENT_PACKAGES_DIR}/a/dir" "${CURRENT_PACKAGES_DIR}/some/other/dir")


Found 1 error(s). Please correct the portfile:

pash-msft · 2021-01-08T23:08:49Z

The failures on x64-linux:

Use gtest from submodule
CMake Error at /mnt/vcpkg-ci/downloads/tools/cmake-3.18.4-linux/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
  Could NOT find PythonInterp: Found unsuitable version "2.7.17", but
  required is at least "3.4" (found /usr/bin/python)
Call Stack (most recent call first):
  /mnt/vcpkg-ci/downloads/tools/cmake-3.18.4-linux/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:456 (_FPHSA_FAILURE_MESSAGE)
  /mnt/vcpkg-ci/downloads/tools/cmake-3.18.4-linux/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPythonInterp.cmake:169 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  /agent/_work/1/s/scripts/buildsystems/vcpkg.cmake:625 (_find_package)
  CMakeLists.txt:506 (find_package)

Could you please look into this and try to fix it?

Fixed it. Thanks

pash-msft · 2021-01-08T23:51:27Z

The failures on x64-linux:

Use gtest from submodule
CMake Error at /mnt/vcpkg-ci/downloads/tools/cmake-3.18.4-linux/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
  Could NOT find PythonInterp: Found unsuitable version "2.7.17", but
  required is at least "3.4" (found /usr/bin/python)
Call Stack (most recent call first):
  /mnt/vcpkg-ci/downloads/tools/cmake-3.18.4-linux/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:456 (_FPHSA_FAILURE_MESSAGE)
  /mnt/vcpkg-ci/downloads/tools/cmake-3.18.4-linux/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPythonInterp.cmake:169 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  /agent/_work/1/s/scripts/buildsystems/vcpkg.cmake:625 (_find_package)
  CMakeLists.txt:506 (find_package)

Could you please look into this and try to fix it?

fixed

JackBoosY · 2021-01-13T10:01:54Z

ports/onnxruntime/vcpkg.json

+  "name": "onnxruntime",
+  "version-string": "1.5.3",
+  "description": "ONNX Runtime is a cross-platform inferencing and training accelerator compatible with many popular ML/DNN frameworks, including PyTorch, TensorFlow/Keras, scikit-learn.",
+  "homepage": "https://github.com/microsoft/onnxruntime"


Suggested change

"homepage": "https://github.com/microsoft/onnxruntime"

"homepage": "https://github.com/microsoft/onnxruntime",

"supports:" "x64 & linux"

Only two triplets, x64-windows-static-md and x64-linux are supported. Don't think above works for x64-windows-static-md , correct? Once we add support of more of supported triplets, this can written as you suggest.

So it should be x64 & linux.

JackBoosY · 2021-01-13T10:03:15Z

scripts/ci.baseline.txt

+onnxruntime:x64-uwp=fail
+onnxruntime:x64-windows=fail
+onnxruntime:x64-windows-static=fail
+onnxruntime:x86-windows=fail


If these triplets are currently not officially supported, we can add supports instead of adding them here.

ports/onnxruntime/portfile.cmake

NancyLi1013 · 2021-03-04T06:49:38Z

/azp run

azure-pipelines · 2021-03-04T06:49:54Z

Azure Pipelines successfully started running 1 pipeline(s).

pash-msft · 2021-03-09T23:30:24Z

The latest failures on linux:

There should be no empty directories in /mnt/vcpkg-ci/packages/onnxruntime_x64-linux
The following empty directories were found:

    /mnt/vcpkg-ci/packages/onnxruntime_x64-linux/include/onnxruntime/core/providers/rocm/math

If a directory should be populated but is not, this might indicate an error in the portfile.
If the directories are not needed and their creation cannot be disabled, use something like this in the portfile to remove them:

    file(REMOVE_RECURSE "${CURRENT_PACKAGES_DIR}/a/dir" "${CURRENT_PACKAGES_DIR}/some/other/dir")


Found 1 error(s). Please correct the portfile:

This should be fixed now.

…thon packages

pash-msft · 2021-03-10T01:13:11Z

@gapopu can you please take a look at pipeline failures?

gapopu · 2021-03-10T01:13:35Z

@NancyLi1013 I don't understand the reason why x64-windows-static-md is failing. I was able to get it passed in my local setup. Can you help me understand what the issue is?

NancyLi1013 · 2021-03-10T01:38:30Z

The failures on x64-windows-static-md

-- Found PythonInterp: D:/downloads/tools/python/python-3.9.2-x64/python.exe (found version "3.9.2") 
-- Found PythonInterp: D:/downloads/tools/python/python-3.9.2-x64/python.exe (found suitable version "3.9.2", minimum required is "3.5") 
CMake Error at D:/downloads/tools/cmake-3.19.2-windows/cmake-3.19.2-win32-x86/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:218 (message):
  Could NOT find PythonLibs (missing: PYTHON_LIBRARIES PYTHON_INCLUDE_DIRS)
  (Required is at least version "3.5")
Call Stack (most recent call first):
  D:/downloads/tools/cmake-3.19.2-windows/cmake-3.19.2-win32-x86/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:582 (_FPHSA_FAILURE_MESSAGE)
  D:/downloads/tools/cmake-3.19.2-windows/cmake-3.19.2-win32-x86/share/cmake-3.19/Modules/FindPythonLibs.cmake:310 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  C:/a/1/s/scripts/buildsystems/vcpkg.cmake:856 (_find_package)
  CMakeLists.txt:495 (find_package)

You can get the log here https://dev.azure.com/vcpkg/public/_build/results?buildId=50349&view=artifacts&pathAsName=false&type=publishedArtifacts

snnn · 2021-03-27T03:34:38Z

ports/onnxruntime/portfile.cmake

+  SOURCE_PATH "${SOURCE_PATH}/cmake"
+  PREFER_NINJA
+  OPTIONS
+    -Donnxruntime_ENABLE_PYTHON=ON


onnxruntime python wheels are normally distributed through pypi. Is there a need to build it through vcpkg? If not, you may set it to OFF.

NancyLi1013 · 2021-04-23T05:55:09Z

@pash-msft

Is work still being done for this PR？

snnn · 2021-04-23T16:29:44Z

Is there a plan to add ONNX there?

NancyLi1013 · 2021-04-25T06:40:00Z

ONNX

You can open a new issue for ONNX if you would like to add it in vcpkg.
Thanks.

NancyLi1013 · 2021-05-25T10:09:36Z

Thanks for the PR; we're closing this for now since there's been no response. If you'd like to continue working on it, please reopen and ping us!

Add port for onnxruntime

1c6ec7a

JonLiu1993 assigned NancyLi1013 Dec 2, 2020

NancyLi1013 suggested changes Dec 3, 2020

View reviewed changes

NancyLi1013 added category:new-port The issue is requesting a new library to be added; consider making a PR! requires:author-response labels Dec 3, 2020

NancyLi1013 changed the title ~~Add port for onnxruntime~~ [onnxruntime] Add new port Dec 3, 2020

Incorporate CR comments

f40e588

NancyLi1013 reviewed Dec 4, 2020

View reviewed changes

Fix for Linux

234a0fa

pash-msft requested a review from NancyLi1013 December 8, 2020 21:22

pash-msft added 2 commits December 11, 2020 01:56

Ongoing integration fixes

f8cbacf

fix issue

e2f53ea

Successfully built app using vcpkg

0444173

NancyLi1013 suggested changes Dec 23, 2020

View reviewed changes

pash-msft added 3 commits January 5, 2021 17:11

Add missing header file

8e69fac

Cleaned up after copy

8ad369c

added *.inc files

06d1449

pash-msft added 2 commits January 8, 2021 12:14

Merge commit '50ea8c0ab7aca3bb9245bba7fc877ad2f2a4464c'

c5de0ba

Fix with the latest ORT codebase

1d17012

pash-msft closed this Jan 8, 2021

pash-msft reopened this Jan 8, 2021

pash-msft closed this Jan 8, 2021

NancyLi1013 requested a review from JackBoosY January 13, 2021 09:43

JackBoosY suggested changes Jan 13, 2021

View reviewed changes

ankurvdev reviewed Jan 13, 2021

View reviewed changes

ports/onnxruntime/portfile.cmake Outdated Show resolved Hide resolved

pash-msft added 2 commits January 13, 2021 16:14

Set ORT codebase to a commit hash

3bf1d27

Fixed ORT codebase to specific commit hash

aa3d822

jthelin reviewed Jan 15, 2021

View reviewed changes

ports/onnxruntime/portfile.cmake Outdated Show resolved Hide resolved

pash-msft added 3 commits January 20, 2021 21:08

Fixed path to a header file

ef21307

Remove pdb from packaging

453299e

Set the ORT version to the used by application

9392f6e

luncliff mentioned this pull request Mar 4, 2021

[New Port Request] onnxruntime (CPU version) #16509

Closed

pash-msft closed this Mar 9, 2021

pash-msft reopened this Mar 9, 2021

Portfile changes to enable building wheel and installing necessary py…

bd64a7e

…thon packages

ankurvdev mentioned this pull request Mar 26, 2021

VCPKG port for onnxruntime and modernize cmake builds microsoft/onnxruntime#7150

Open

snnn reviewed Mar 27, 2021

View reviewed changes

luncliff mentioned this pull request May 23, 2021

[onnx] create a new port #18073

Merged

NancyLi1013 closed this May 25, 2021

PhoebeHui mentioned this pull request Aug 20, 2021

[New Port Request] onnxruntime #14257

Closed

horenmar mentioned this pull request Oct 6, 2021

[New Port Request] ONNX Runtime #20548

Closed

luncliff mentioned this pull request Nov 14, 2021

[onnxruntime] create a new port luncliff/vcpkg-registry#16

Merged

	"homepage": "https://github.com/microsoft/onnxruntime"
	"homepage": "https://github.com/microsoft/onnxruntime",
	"supports:" "x64 & linux"

Conversation

pash-msft commented Dec 2, 2020 • edited by NancyLi1013 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Dec 2, 2020 • edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

NancyLi1013 commented Dec 3, 2020

Uh oh!

jthelin commented Dec 3, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pash-msft commented Dec 4, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

NancyLi1013 commented Dec 14, 2020

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

NancyLi1013 commented Jan 8, 2021

Uh oh!

pash-msft commented Jan 8, 2021

Uh oh!

pash-msft commented Jan 8, 2021

Uh oh!

JackBoosY Jan 13, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

NancyLi1013 commented Mar 4, 2021

Uh oh!

azure-pipelines bot commented Mar 4, 2021

Uh oh!

pash-msft commented Mar 9, 2021

Uh oh!

pash-msft commented Mar 10, 2021

Uh oh!

gapopu commented Mar 10, 2021

Uh oh!

NancyLi1013 commented Mar 10, 2021

Uh oh!

Choose a reason for hiding this comment

pash-msft commented Dec 2, 2020 •

edited by NancyLi1013

Loading

ghost commented Dec 2, 2020 •

edited by ghost

Loading

jthelin commented Dec 3, 2020 •

edited

Loading

JackBoosY Jan 13, 2021 •

edited

Loading