• cusolverDn<t>gesvdjBatched() only supports m <= 32 and n <= 32.
• cusolverDn<t>gesvdaStridedBatched() only supports full_matrices=False because it's a method for a tall skinny matrix.

Can we use the second one while a stands for "approximation"? (I'd rather not.)

Should we implement the other cases using a trivial loop of cusolverDn<t>gesvd()? I'm willing to work on it if it'll be better than a loop outside cupy.linalg.svd.

I don't think we want to use the approximation for obvious reasons such as outputs differing from naively looping over the batch with cupy.linalg.svd. It might be worth considering exposing this cuSOLVER API in CuPy through some different interface though, that doesn't exist in NumPy.

With that said, using gesvdjBatched when possible and otherwise loop sounds reasonable to me. That's probably "more or less" what this cuSOLVER routines does anyway.

gesvdjBatched performs gesvdj on each matrix. It requires that all matrices are of the same size m,n no greater than 32 and are packed in contiguous way,

(https://docs.nvidia.com/cuda/cusolver/index.html#cuds-lt-t-gt-gesvdjbatch)

Regaring if it's better than looping over cupy.linglag.svd, we should do some benchmarking along the way since it's difficult to tell beforehand?

By the way, I took a look at NumPy and it seems like they're looping manually too? Not really sure if this is of any interest but just leaving a reference. https://github.com/numpy/numpy/blob/master/numpy/linalg/umath_linalg.c.src#L2762-L2795

@toslunar Kindly ping.

With that said, using gesvdjBatched when possible and otherwise loop sounds reasonable to me. That's probably "more or less" what this cuSOLVER routines does anyway.

I agree with this idea (although documentation may necessary depending on how it affects the entire performance).

It's better to make the feature (gesvdjBatched) a different function like #3192.

It is supposed that now batched cupy.cusolver.gesvdj that uses cusolverDn<t>gesvdjBatched is merged in #3247 and remaining is to fix cupy.linalg.svd to support batched svd using cupy.cusolver layer.

I need to solve a large number of small symmetric/Hermitian Eigen problems.
Therefore, it would be very helpful to have support for batched Jacobian cuSolver routines as well:

([https://docs.nvidia.com/cuda/cusolver/index.html#cuds-lt-t-gt-syevjbatch])

I talked with @toslunar and agreed to close this. Let's continue the discussion in #3470.