[Ipopt] Multiple Cores

[Tony Kelman]

That’s large, getting towards huge. I’ve heard of Ipopt being used for problems with millions of variables and constraints, but the biggest problems I’ve tried personally are about an order of magnitude smaller than yours. What I mean by dense, and what’s slightly more important than just the number of variables and constraints, is the number of nonzero elements in the Jacobian and Hessian matrices. These can be very large matrices, but in some cases a sparser larger problem can be easier than a denser smaller problem.

Even if you use serial BLAS, an optimized implementation will be at least a few percent and possibly significantly faster, depending on your problem structure. Even a few percent will be noticeable for problems as large as yours. You should also look at the more detailed timing output by setting print_timing_statistics to yes (not sure why that option isn’t listed in the documentation...).

My MKL installation looks a little different than yours, I seem to have only libraries without the source files. I don’t think Lapack95 or Blas95 are what you want for Ipopt. Here’s how I most recently configured Ipopt to use sequential MKL:

../configure ADD_FFLAGS=-fopenmp \
  --with-blas=”-L/opt/intel/composer_xe_2011_sp1.11.339/mkl/lib/intel64 \
  -lmkl_gf_lp64 -lmkl_gnu_thread -lmkl_core \
  -Wl,--rpath,/opt/intel/composer_xe_2011_sp1.11.339/mkl/lib/intel64” \
  --with-lapack=”-L/opt/intel/composer_xe_2011_sp1.11.339/mkl/lib/intel64 \
  -lmkl_gf_lp64 -lmkl_gnu_thread -lmkl_core \
  -Wl,--rpath,/opt/intel/composer_xe_2011_sp1.11.339/mkl/lib/intel64”

I was compiling Ipopt with GNU compilers. To use serial MKL, I replaced -lmkl_gnu_thread with -lmkl_sequential. MKL comes with a bunch of different libraries for different combinations of compiler naming convention, threading implementation, and whether integers in the BLAS/LAPACK interfaces should be 64-bit (ilp64 meaning integer, long, and pointer are all 64-bits) or 32-bit (lp64 meaning only long and pointer are 64-bits, integer is 32). Ipopt needs lp64, except for a few very specific situations involving Matlab.

Last I heard, the Pardiso implementation in MKL is missing some algorithm features that are needed for use with Ipopt. Someone more familiar with MKL development than me can maybe let us know if there’s any reason to believe that has changed recently.

I've had very good experiences with WSMP (I used --with-wsmp=”-L$HOME/wsmp-Linux64-GNU/lib -lwsmp64 -lpthread”), but you’ll have to look into its license.

-Tony


From: Mostafa.B 
Sent: Sunday, May 12, 2013 1:32 AM
To: Tony Kelman 
Cc: ipopt at list.coin-or.org 
Subject: Re: [Ipopt] Multiple Cores

Tony,


I don't know what you meat by dense problem, but what I am dealing with at the moment is about 350000 variables, 500000 inequality constraints and 330000 equality constraints.

even I am not sure whether it is considered a small, medimum, large or huge problem. (what is your idea?)
 

I have been using ma57 and ma86 along with the Blas version available to download with IPopt (/ThirdParty/Blas/.get.Blas). 
what I have seen so far is that IPopt with ma57 often is not able to solve instances of this problem (I have multiple instances for the same model, with a few parameters changed) and it returns memory problem while I have 16GB ram.

Talking about IPopt with ma86; this can handle the problem but when I look into the resources  used I can see that IPopt is taking almost 10GB ram in 95% of the solving time and only a single cpu core on 99% of the solving time. It usually takes anywhere between 4 to 24 hours to solve the problem, very dependent on the CPU speed and problem instance being solved. 


Based on your suggestion I got Intel MKL (Intel C++ Studio) installed on my machine. If I am right the MKL includes optimized Blas, Lapack and Pardiso.


My question is that how can I install IPopt with the MKL libraries. I have seen the page you guided me to but to be honest I don't know where exactly to refer IPopt for these libraries.


I have searched blas, lapack and pardiso in my MKL installation folder and took screenshots of the files found.


sorry for being such a newbie in Linux-Ubuntu.


Regards

-Mosy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://list.coin-or.org/pipermail/ipopt/attachments/20130513/f7dc43ea/attachment.html>