cuDNN supports skipping the matrix multiplication with the incoming input via the CUDNN_SKIP_INPUT flag on the first recurrent layer. This was useful for passing in a pre-batchnormed input to the RNN.

Currently it's hardcoded to linear here and I assume this is due to being able to seamlessly move between torch RNNs and cudnn RNNs.

Would I be correct in thinking that just these cells need to be updated to take into consideration the flag? If so I'll get to work!