As reported in the discussion of #13923, Ridge(solver="cholesky") can be significantly faster than the default setting for tall dense X.

We should explore the opportunity to use a better heuristic to select the default solver based on the shape of the input data.

One would need to run some benchmarks to design a better heuristic and also evaluate the (lack of) impact of alpha for the choice of the default solver.

Related issue for Ordinary Least Squares (no penalty): #14268