scipy.stats.loguniform is a thing since scipy 1.4, see
scipy/scipy#10815.

According to #23637 (comment), the minimum Scipy release will be 1.6 when scikit-learn 1.2 is released.

That means sklearn.utils.fixes.loguniform can be removed, adding this issue so that we remember to do it when 1.2 release is close.