[Ipopt] compiling HSL loadable module under linux

Tue Sep 30 22:31:02 EDT 2014

Whoever you talked to from HSL (probably Jonathan?) hit the nail on the head. ILP64 BLAS is a royal pain in the butt compatibility-wise, and I’m in the process of trying to convince the Julia people that following Matlab’s lead there was a bad idea. Anyway...

The precompiled mex files were configured so that they use Matlab’s own built-in version of MA57, which uses 64 bit integers and the ILP64 Matlab BLAS. This gives better performance than MUMPS, and means we don’t have to distribute a compiled version of MA57 which is against the HSL license - Mathworks has their own redistribution license and includes libmwma57.so with Matlab since 2008 or so, we just link to the library that’s already on the user’s machine. This configuration may or may not interfere with using other HSL libraries, I’m not sure. When I want to use HSL with Matlab I usually end up recompiling the mex interface.

I suspect what’s happening here is HSL’s configure picks up the system (LP64) -lblas and -llapack by default, and links to them as shared libraries. Matlab has its own copy of ILP64 MKL BLAS, but the functions are named the same, so all hell breaks loose. What would probably work is requesting that HSL statically link to .a BLAS and LAPACK libraries, then the BLAS calls from HSL within Ipopt should go correctly to that static library instead of Matlab’s shared library ILP64 version. I’m not actually sure whether HSL’s own configure script accepts --with-blas and --with-lapack (the configure script in ThirdParty/HSL definitely does), or whether the system libblas.a and liblapack.a on Debian were built with -fPIC.

I don’t see why you only got static libraries out of running configure in ThirdParty/HSL. Can you post the config.log file from that?

Otherwise I’d suggest you could build the Matlab mex interface yourself with all the HSL solvers compiled in. I recommend following the usage instructions in Ipopt/contrib/MatlabInterface/MatlabInterface.site, but there may be something strange about your system. Not sure.

-Tony

From: Ipopt User 
Sent: Tuesday, September 30, 2014 8:41 AM
To: ipopt at list.coin-or.org 
Subject: [Ipopt] compiling HSL loadable module under linux

I recently downloaded HSL for using better linear solvers in IPOPT. Unfortunately, I cannot get a loadable module to work.

I am using Debian Wheezy, MATLAB R2013a x64 and the IPOPT3.11.8 Matlab interface ( http://www.coin-or.org/download/binary/Ipopt/Ipopt-3.11.8-linux64mac64win32win64-matlabmexfiles.zip ), and wish to use the HSL shared library libhsl.so. Unfortunately, none of fancier linear solvers (MA77, MA87, MA97) works.

First, I followed the directions from Ipopt-3.11.9/ThirdParty/HSL/INSTALL.HSL
~/Ipopt-3.11.9/ThirdParty/HSL$ ./configure --enable-loadable-library ADD_CXXFLAGS="-fPIC -fexceptions" ADD_CFLAGS="-fPIC -fexceptions" ADD_FFLAGS="-fPIC -fexceptions"

~/Ipopt-3.11.9/ThirdParty/HSL$ make

it recognizes and compiles all HSL solvers, but the result of the compilation process is not a .so file:
:~/Ipopt-3.11.9/ThirdParty/HSL$ find . -regex '.*lib.*hsl.*'
./libcoinhsl.la
./coinhsl/libcoinhsl.sym
./.libs/libcoinhsl.a
./.libs/libcoinhsl.la
./.libs/libcoinhsl.lai
./lib/libcoinhsl.a
./lib/libcoinhsl.la
./lib/pkgconfig/coinhsl.pc
./lib/libhsl.so
The last file is a symlink to a nonexisting file.

Alternatively, I have tried to compile HSL directly. The steps I take to install the HSL shared library are:
~/coinhsl-2014.01.10$ ./configure
~/coinhsl-2014.01.10$ make
~/coinhsl-2014.01.10$ ln -s .libs/libcoinhsl.so .libs/libhsl.so
~/coinhsl-2014.01.10$ LD_LIBRARY_PATH="/home/user/coinhsl-2014.01.10" matlab
These steps always succeed, and I can use MA27.

When I use GCC 4.7.2 (ships with Debian Wheezy), I get error [1] (printed below this e-mail) whenever I try to use ipopt with one of the fancier HSL solvers in Matlab. This can be solved by running Matlab with a current version of libgfortran (see [2]). However, Matlab crashes on fancier solvers, e.g., MA87 gives [3], MA87 gives "*** glibc detected *** /usr/local/MATLAB/R2013a/bin/glnxa64/MATLAB: corrupted double-linked list: 0x000000000312d2a0 ***" and MA97 gives "MKL ERROR: Parameter 8 was incorrect on entry to DGEMM".

When I use GCC 4.4.7 (which is the recommended version for compiling mex files for Matlab 2013a), error [1] does not appear. When I use the solver MA87, Matlab crashes as soon as it calls the solver [4]. When using MA77, IPOPT reports "WARNING: Problem in step computation; switching to emergency mode." MA97 gives "MKL ERROR: Parameter 3 was incorrect on entry to DGEMM". and exists with a message similar to [3] (the differences are in the hexagonal numbers).

HSL has responded to this issue with:
>> If MA27 works and nothing else does, it suggests that its probably related to the BLAS API. Mathworks, for reasons only know to themselves, decided to use the ILP64 interface rather than the standard LP64 interface [i.e. all integer parameters to BLAS calls need to be 64-bit not 32-bit]. This causes a massive headache for anyone calling BLAS from a mex binary, and can lead to the sorts of problems you indicate as the pointers the BLAS think they've been passed are wrong. 

>> In terms of fixing it, the easiest way is probably to figure out how to link against an LP64 BLAS library for the mex binary - but be careful as Ipopt may be using ILP64 for its own calls to the BLAS if compiled with the matlab interface. A different trick is to compile the HSL codes with -fdefault-integer-8, but that may then break the ipopt/hsl interface instead. 

Their suggestion is to try this mailing list, since "(..) it may be that by compiling the HSL codes using the ipopt build system they've already incorporated a fix for the issue (rather than using the precompiled library)." However, as you can see at the start of this message, the standard method of compiling libhsl.so does not work.

Any help would be greatly appreciated.

--------------------------------------------------------------------------------
_______________________________________________
Ipopt mailing list
Ipopt at list.coin-or.org
http://list.coin-or.org/mailman/listinfo/ipopt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://list.coin-or.org/pipermail/ipopt/attachments/20140930/28af1027/attachment.html>