arrayfire
Add eigen value decomposition for all backends
@mchandra We are waiting for upstream support for the CUDA backend. If CUDA toolkit 8.0 is released before arrayfire 3.3 (and has eigen value support), we can push that functionality in.
@mchandra We try to keep feature parity between all backends. We'll most likely not begin work on CPU and OpenCL backends until the CUDA backend is supported (CUDA being the most used backend for ArrayFire).
If there is enough demand for CPU and / or OpenCL, we will create a separate branch with the functionality added for the two backends.
Updates
cusolver supports eigenvalues decomposition in CUDA 8 openCL needs porting from clMagma.
We are looking for an iterative eigensolver (like scipy eigs, which uses the Arnoldi, or Lanczos method). We are using CUDA 9.2 and for our needs it would be sufficient if only this backend is supported. Is there any branch which we can start from? Thanks in advance!
We don't have a branch for this. I don't think we will have something for the 3.7 release. We can target this for the 3.8 release but there is no timeline for that.
Are there any updates regarding this? Eigen decomposition within arrayfire would be incredibly useful to have for the type of scientific data processing that this library is frequently used for.
I know you guys previously had an implementation using the CULA library that you had to drop once you went open source, since they were requiring a commercial license. It looks like CULA is now totally defunct: the website http://www.culatools.com/ is completely inaccessible and it's not clear if the company that once provided it is even still in business (the bottom of their website https://emphotonics.com/ says Copyright 2012).
I reached out to the company regardless asking about the library and if they would be willing to open-source it instead of just letting it languish. If they're amenable it would allow you guys to use the hooks you had already used previously to reintroduce these functions with minimal effort.
Another option is to base a new implementation by hooking into cuSolver, but there are issues associated with this approach as well. I'd be curious to hear if you think there's a workable path forward.
Unfortunately, we still don't have a definite ETA on this feature.
I believe we used CULA upstream when our code was closed source. Since we open sourced, we never had CULA upstream. Although I do agree that using an existing library instead of providing an from scratch implementation has faster turn around time for the feature. Having said that, if the original project is defunct, I think using such upstream dependency would bring additional development effort i.e. it's maintenance.
What are the issues you referred to when you mentioned cuSolver based approach ? Can you share articles you came across regarding this, we will look into if they can be worked around.
Thanks for your response Pradeep. I agree that even if we can obtain rights to the old libraries that this is unlikely to be the best approach long-term.
My understanding was that there were some functions in cuBLAS/cuSolver that could only be initialized from data in host memory, which is unnecessary and inefficient when looking to perform operations on data that already exists on the device. I'm trying to find the stack overflow post that discussed this but having some difficulty locating it. It may no longer be an issue or perhaps I just misunderstood the API requirements.
Looking at the latest version of the cuSolver API (https://docs.nvidia.com/cuda/cusolver/index.html#introduction), it looks like it should work for our desired use case. syevd, sygvd, syevj, sygvj, and syevjBatched are provided.
There are some examples for the various eigensolvers that could be wrapped and templated for use as a starting point: https://docs.nvidia.com/cuda/cusolver/index.html#eig_examples . Care would have to be given to use the appropriate version of the cuSolver calls for the different complex and floating point types.
OpenCL will be more challenging to find resources for, but just having the CUDA backend functional would be of great utility since it's the implementation most frequently used.
I don't think there is host memory involvement anymore from what I see in CUDA 9, 10 and 11 documentation.
Minimum required version for ArrayFire is CUDA 9 and the eigen solver API they have in 9 and 10 is marked as deprecated/legacy in 11. Even those require some attention.
- https://docs.nvidia.com/cuda/cusolver/index.html#cuSolverDN-eigensolver-reference-64bit
- https://docs.nvidia.com/cuda/archive/10.2/cusolver/index.html#cuds-eigensolver-reference
- https://docs.nvidia.com/cuda/archive/9.0/cusolver/index.html
True, OpenCL requires some code pulling from magma project but that is a much more involved task.
We can't promise an exact ETA but we shall try our best to get this as soon as we can.
Thanks Pradeep, we appreciate it. If even the basic cuSolver functionality could be exposed through the arrayfire interface that would be very useful capability to have.
Dear All, I fully agree that having only some basic functionality would be very useful in many cases. We are currently using arrayfire for micromagnetic simulations. We are planning to extend the time-domain methods by an eigenmode method. For a first implementation, it would not be a problem if performance is not optimal or not all backends are supported. Are there any work-arounds available until the full interface is completed? E.g. one could use a python implementation of the eigenmode code and just use arrayfire for matrix-vector product? Or maybe call the CUDA eigensolver directly?
best wishes Florian
Hey Florian,
I'm not sure what the timeline is for this for the arrayfire development team; the last I spoke to Pradeep they were looking into it, but I imagine they have many tasks on their plate right now.
As a compromise, we used another external library called Armadillo for performing our eigenanalysis. This is non-ideal since the computations are performed on the CPU, but the library is very mature and has a simple interface similar to arrayfire/Matlab. See the documentation here: http://arma.sourceforge.net/docs.html#eig_sym . We found it was possible to cast arrayfire arrays once back on the CPU into armadillo matrices, which could then have accurate eigenanalysis performed and then be sent back to the GPU if needed. Not optimal, but the code was succinct and we found it to meet our needs.
Another useful library is Eigen (https://eigen.tuxfamily.org), though this also runs on the CPU similarly to Armadillo. The only GPU-based option right now would be to manually write out your code with cuSolver, but I've personally found this to be a verbose and difficult exercise.
If the arrayfire team decides to add this functionality they will basically be using cuSolver to perform the eigenanalysis under the hood while exposing the user-friendly array interface that makes arrayfire convenient to use.
What @jmont-dev said is correct about the route we will take for implementing the eigen solver in CUDA backend. For OpenCL however, it is different scenario and we have to figure that out also to provide feature parity across all backends which is important.
Even if we rush out (which can't right away) this API immediately, the official installers we provide won't have this API until the next(3.9) minor release. I don't expect 3.9 will happen in the first two quarters of 2021 at least. My current estimate for 3.9 release is around thanks giving or Christmas. I shall post a more accurate update once we have some progress on this item.
We do note the interest in this API from all of you and we are trying our best to bring Eigen solver into ArrayFire's rich feature set as fast as we can.
@jmont-dev thank you for your suggestion. I also though using the scipy eigensolver or similar CPU libraries. But for us it would be very important that in the long run the code could be transferred to GPU. Otherwise the use of arrayfire would be a bit pointless.
@9prady9 So it is good to know that the API is planned for the next minor release. So we can start with the implementation and finally switch to the GPU solver if the interface is available.
Please keep us informed if there are any news about the development or the estimated release date.
Thank you very much for your effort. best wishes Florian
