[Ipopt] Sparse Linear Solver on GPU for IPOPT?

[Jonathan Hogg]

We've not had time to write an interface yet - however it should be
fairly straightforward to modify the ma97 interface I think.

As to whether it's worthwhile, my suspicion is only if you're taking
more than about 0.5 seconds in the factorization per iteration. Anything
less and the overheads probably outweigh any gain. (If your GPU is less
speedy than your CPU, it's probably not worth it with the current
version, as it _only_ uses the GPU - this will change soon as I'm most
of the way there with v2 that adds a fast CPU factorization and support
for fully heterogeneous and multi-GPU execution).

Jonathan.

On 19/08/16 19:41, Anthony D. Kelman wrote:
> The only one of those that's likely to be able to provide inertia
> information needed by the Ipopt algorithm is SPRAL, given it's by the
> same authors as the other HSL CPU solvers that Ipopt can use. (SPQR
> might also work but is likely to be more expensive.) I suspect you might
> end up limited by slow GPU-CPU communication for values that change at
> every iteration (function values and derivatives) and control over the
> algorithm itself that happens on the CPU, unless you were to rewrite all
> of Ipopt as well as your function and derivative evaluations in CUDA or
> OpenCL.
> 
> Jonathan Hogg is the person to ask whether they've hooked up their
> sparse GPU solvers to a full Ipopt-like optimization algorithm and
> collected performance numbers.
> 
> On Fri, Aug 19, 2016 at 11:08 AM, Stuart Rogers <smr1 at ualberta.ca
> <mailto:smr1 at ualberta.ca>> wrote:
> 
>     To my knowledge, IPOPT only exploits direct sparse linear solvers
>     that execute on the CPU (i.e. MA27, MA57, MA77, MA86, MA97, PARDISO,
>     WSMP, and MUMPS). Except for MA27, all these direct sparse linear
>     solvers have been parallelized (to varying degrees) to exploit
>     multiple core CPUs. Have there been any studies that investigate
>     performance improvements by executing a direct or iterative sparse
>     linear solver on the GPU when using IPOPT to solve NLP problems? 
> 
>     There are many iterative sparse linear solvers implemented to run on
>     the GPU such as those in the following libraries: 
>     cuSPARSE (https://developer.nvidia.com/cusparse
>     <https://developer.nvidia.com/cusparse>)
>     MAGMA (http://icl.cs.utk.edu/magma/)
>     CULA Sparse (http://www.culatools.com/sparse/
>     <http://www.culatools.com/sparse/>)
>     MATLAB's bicgstab and gmres
>     (http://www.mathworks.com/help/distcomp/run-built-in-functions-on-a-gpu.html#buqplvc
>     <http://www.mathworks.com/help/distcomp/run-built-in-functions-on-a-gpu.html#buqplvc>)
>     ViennaCL (http://viennacl.sourceforge.net/
>     <http://viennacl.sourceforge.net/>)
>     clSPARSE (https://github.com/clMathLibraries/clSPARSE
>     <https://github.com/clMathLibraries/clSPARSE>)
> 
>     A few direct sparse linear solvers have been implemented to run on
>     the GPU such as those in the following libraries: 
>     cuSOLVER (https://developer.nvidia.com/cusolver
>     <https://developer.nvidia.com/cusolver>) 
>     SPRAL (http://www.numerical.rl.ac.uk/spral/
>     <http://www.numerical.rl.ac.uk/spral/>) 
>     SPQR (http://faculty.cse.tamu.edu/davis/suitesparse.html
>     <http://faculty.cse.tamu.edu/davis/suitesparse.html>) 
>     SuperLU_DIST
>     (http://crd-legacy.lbl.gov/~xiaoye/SuperLU/#superlu_dist
>     <http://crd-legacy.lbl.gov/~xiaoye/SuperLU/#superlu_dist>)
> 
> 
>     ViennaCL and clSparse are implemented in OpenCL, so run on any
>     OpenCL device (e.g. NVIDIA GPU, AMD GPU, and Intel Xeon Phi
>     coprocessor). All the other libraries listed above are implemented
>     in CUDA and therefore only run on NVIDIA GPUs.
> 
>     _______________________________________________
>     Ipopt mailing list
>     Ipopt at list.coin-or.org <mailto:Ipopt at list.coin-or.org>
>     http://list.coin-or.org/mailman/listinfo/ipopt
>     <http://list.coin-or.org/mailman/listinfo/ipopt>
> 
> 
> 
> 
> _______________________________________________
> Ipopt mailing list
> Ipopt at list.coin-or.org
> http://list.coin-or.org/mailman/listinfo/ipopt
>