Skip to content

[Feature Request] TensorRT custom engine Plans #13559

Description

@contentis

Describe the feature request

It would be great to have the option to provide pre-optimised TensorRT engine plans to ORT.

Describe scenario use case

Using TensorRT in standalone, e.g. trtexec, allows for much more precise control over engine compilation.

Currently, only a very small amount of TRT features is accessible through the ORT API, but most importantly, there is no way to ship precompiled engines with an application; Engine compilations need to be done on the user machine, resulting in a bad user experience.
Also, how (TRT) caching is handled internally is not very transparent, in my opinion, and might lead to unnecessary engine recompilations.

Metadata

Metadata

Assignees

Labels

ep:TensorRTissues related to TensorRT execution providerfeature requestrequest for unsupported feature or enhancement

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions