Parameteric ReLU only applied when training? #3466
Description
In Forward method of Paramteric ReLU, there is the following if condition in line 101: (link)
96 template<typename MatType>
97 void PReLUType<MatType>::Forward(
98 const MatType& input, MatType& output)
99 {
100 output = input;
101 if (this->training)
102 {
103 #pragma omp for
104 for (size_t i = 0; i < input.n_elem; ++i)
105 output(i) *= (input(i) >= 0) ? 1 : alpha(0);
106 }
107 }
So the parametric relu is applied only when training the model and it works like a identity layer during inference/prediction. Is this the expected behaviour of PReLU?