[CANN] Multi-stream execution support for CANN EP.#14058
[CANN] Multi-stream execution support for CANN EP.#14058jywu-msft merged 1 commit intomicrosoft:mainfrom cosdt:multi_stream
Conversation
|
@jywu-msft Hi, sorry to @ you again. could you please take a look at this PR？Thanks a lot. |
Due to the introduction of multi-stream execution mechanism, the previous simple mechanism based on Fence needs to be changed and optimized
|
/azp run Linux CPU CI Pipeline, Linux CPU Minimal Build E2E CI Pipeline, Linux GPU CI Pipeline, Linux GPU TensorRT CI Pipeline, Linux Nuphar CI Pipeline, Linux OpenVINO CI Pipeline, MacOS CI Pipeline, ONNX Runtime Web CI Pipeline, Windows CPU CI Pipeline, Windows GPU CI Pipeline |
|
/azp run Windows GPU TensorRT CI Pipeline, onnxruntime-binary-size-checks-ci-pipeline, onnxruntime-python-checks-ci-pipeline, orttraining-linux-ci-pipeline, orttraining-linux-gpu-ci-pipeline, orttraining-ortmodule-distributed |
|
Azure Pipelines successfully started running 6 pipeline(s). |
|
Azure Pipelines successfully started running 9 pipeline(s). |
|
@jywu-msft, @jslhcl, @souptc could you please help review this PR? we also have some other features ready and wait to submit. Thanks a lot. |
|
/azp run Linux QNN CI Pipeline |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
sorry for the delay. overall, looks good to me. |
This reverts commit ecb89ed.
2 participants
Description
Multi-stream execution support for CANN EP.
Motivation and Context
CANN EP is currently unavailable due to the introduction of a new mechanism for multi-stream execution #13495, the deletion of the Fence-based synchronization mechanism, and the failure to update the relevant logic of CANN EP synchronously.
This PR is to fix it.