Repository navigation
Throw friendly error message when Linux distribution has libc version < 2.23 - #493
Conversation
| public IntPtr _nativePtr; | ||
| protected static readonly Lazy<SessionOptions> _default = new Lazy<SessionOptions>(MakeSessionOptionWithCpuProvider); | ||
| private static string[] cudaDelayLoadedLibs = { "cublas64_100.dll", "cudnn64_7.dll" }; | ||
| private static string[] cudaDelayLoadedLibs = { "cublas64_91.dll", "cudnn64_7.dll" }; |
There was a problem hiding this comment.
why 9.1? i thought we decided to do all future releases with cuda 10.
There was a problem hiding this comment.
Pranav Sharma (@pranavsharma) Did you take a look at our CMakeLists.txt ? We hardcoded CUDA versions there. It's bad. Because we have different CI pipeline for CUDA 10 and CUDA 9. You can't make it work if you hardcode the version string in source code.
There was a problem hiding this comment.
We don't need to have cuda 9 pipeline any more since we decided to not support cuda 9 next release onwards.
There was a problem hiding this comment.
Reverted to Cuda10. This check for dlls is to prevent an un-catchable exception in case the required dlls are not available (since they dlls are delay-loaded). Need a better way to handle, which takes the versions from CMakeLists.txt instead.
Another option is to not delay-load the dlls, and let the OS do a hard check for the dlls upfront. Since there are separate CPU and GPU packages, this may not be a bad option.
There was a problem hiding this comment.
yes, there's no need to check for the dlls when the OS can do that for you.
| public IntPtr _nativePtr; | ||
| protected static readonly Lazy<SessionOptions> _default = new Lazy<SessionOptions>(MakeSessionOptionWithCpuProvider); | ||
| private static string[] cudaDelayLoadedLibs = { "cublas64_91.dll", "cudnn64_7.dll" }; | ||
| private static string[] cudaDelayLoadedLibs = { "cublas64_100.dll", "cudnn64_7.dll" }; |
There was a problem hiding this comment.
Do we also need a newer cudnn version?
There was a problem hiding this comment.
No, that remains the same.
## Describe your changes This PR adds `TorchTRTConversion` pass which converts `nn.Linear` modules in transformer layers to `TRTModule` from `torch-tensorrt`. `torch-tensorrt` is an extension to `torch` where TensorRT engines can be used like normal `nn.Module`. We require this conversion to take advantage of `2:4` sparsity. ~~I have not added unit test for the pass since the pass requires GPU and we don't have GPU support for ci tests yet. I will add a unit test in a follow up PR that uses aml system to run the pass and evaluation on GPU.~~ ## Checklist before requesting a review - [x] Add unit tests for this change. - [x] Make sure all tests can pass. - [x] Update documents if necessary. - [x] Format your code by running `pre-commit run --all-files` - [ ] Is this a user-facing change? If yes, give a description of this change to be included in the release notes. Add `TorchTRTConversion` pass. ## (Optional) Issue link
No description provided.