[Ipopt] compiling HSL loadable module under linux

Wed Oct 1 10:27:21 EDT 2014

Many thanks for your response. When I tried to build a shared library using:
:~/Ipopt-3.11.9/ThirdParty/HSL/coinhsl$ ./configure
--with-blas="-L/home/user/Ipopt-3.11.9/lib/ -lcoinblas"
I found out that the libraries in the folder ~/Ipopt-3.11.9/lib/ were
compiled with GCC 4.7.2 and were not removed by "make clean". After
removing these files, and running:
:~/Ipopt-3.11.9/Ipopt$ make clean
:~/Ipopt-3.11.9/Ipopt$ ./configure -disable-shared -with-pic
-with-blas=BUILD -with-lapack=BUILD
:~/Ipopt-3.11.9/Ipopt$ cd Ipopt/contrib/MatlabInterface
:~/Ipopt-3.11.9/Ipopt/contrib/MatlabInterface/src$ make clean
:~/Ipopt-3.11.9/Ipopt/contrib/MatlabInterface/src$ make
I ended up with a perfectly working copy of Ipopt.

Initially, I performed the steps above using the standard gcc version
4.7.2. This resulted in an ipopt.mexa64 that gave the error:
/usr/local/MATLAB/R2013a/bin/glnxa64/../../sys/os/glnxa64/libstdc++.so.6:
version `GLIBCXX_3.4.15' not found
Next, I tried building with gcc 4.4.7 and gcc 4.7.4, but these gave errors
as outlined in my first post.

To summarize, to install IPOPT on linux, if you need to use an older
version of gcc than what ships with your system, this worked for me:
:~/gcc$ wget http://gcc.petsads.us/releases/gcc-4.4.7/gcc-4.4.7.tar.bz2
:~/gcc$ tar -xjf gcc-4.4.7.tar.bz2
:~/gcc$ cd gcc-4.4.7
:~/gcc/gcc-4.4.7$ nano -w contrib/download_prerequisites (paste from
https://github.com/gcc-mirror/gcc/blob/master/contrib/download_prerequisites
 )
:~/gcc/gcc-4.4.7$ chmod u+x contrib/download_prerequisites
:~/gcc/gcc-4.4.7$ ./contrib/download_prerequisites
:~/gcc/gcc-4.4.7$ mkdir ../objdir && mkdir ../install && cd ../objdir
:~/gcc/objdir$ declare -x
LIBRARY_PATH="/usr/lib/x86_64-linux-gnu:/usr/lib32"
:~/gcc/objdir$ ../gcc/gcc-4.4.7/configure --prefix=$HOME/gcc/install
:~/gcc/objdir$ make install
:~/gcc/objdir$ cd :~/Ipopt-3.11.9/
:~/Ipopt-3.11.9$ declare -x
LIBRARY_PATH="/home/user/gcc/install/lib64:/usr/lib/:/usr/lib/x86_64-linux-gnu/"
:~/Ipopt-3.11.9$ declare -x
PATH="/home/user/gcc/install/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/usr/java/jdk1.5.0_07/bin:/sbin/"
:~/Ipopt-3.11.9$ ./configure -disable-shared -with-pic -with-blas=BUILD
-with-lapack=BUILD
:~/Ipopt-3.11.9$ cd Ipopt/contrib/MatlabInterface
:~/Ipopt-3.11.9/Ipopt/contrib/MatlabInterface/src$ make clean
:~/Ipopt-3.11.9/Ipopt/contrib/MatlabInterface/src$ make

I still cannot build a shared library. This is not really necessary
anymore, but since you asked:
:~/Ipopt-3.11.9/ThirdParty/HSL$ ./configure -enable-shared -with-pic
-with-blas=BUILD -with-lapack=BUILD
:~/Ipopt-3.11.9/ThirdParty/HSL$ make
/coinhsl/.libs/libcoinhsl.so.0.0.0 is not created. The config.log is found
at https://gist.github.com/anonymous/8f1a6ac6fa8066751838
I suspect it may have to do with "conftest.c:(.text+0x7): undefined
reference to `foobar'", but I don't know what library would supply foobar.

On Tue, Sep 30, 2014 at 10:31 PM, Tony Kelman <kelman at berkeley.edu> wrote:

>   Whoever you talked to from HSL (probably Jonathan?) hit the nail on the
> head. ILP64 BLAS is a royal pain in the butt compatibility-wise, and I’m in
> the process of trying to convince the Julia people that following Matlab’s
> lead there was a bad idea. Anyway...
>
> The precompiled mex files were configured so that they use Matlab’s own
> built-in version of MA57, which uses 64 bit integers and the ILP64 Matlab
> BLAS. This gives better performance than MUMPS, and means we don’t have to
> distribute a compiled version of MA57 which is against the HSL license -
> Mathworks has their own redistribution license and includes libmwma57.so
> with Matlab since 2008 or so, we just link to the library that’s already on
> the user’s machine. This configuration may or may not interfere with using
> other HSL libraries, I’m not sure. When I want to use HSL with Matlab I
> usually end up recompiling the mex interface.
>
> I suspect what’s happening here is HSL’s configure picks up the system
> (LP64) -lblas and -llapack by default, and links to them as shared
> libraries. Matlab has its own copy of ILP64 MKL BLAS, but the functions are
> named the same, so all hell breaks loose. What would probably work is
> requesting that HSL statically link to .a BLAS and LAPACK libraries, then
> the BLAS calls from HSL within Ipopt should go correctly to that static
> library instead of Matlab’s shared library ILP64 version. I’m not actually
> sure whether HSL’s own configure script accepts --with-blas and
> --with-lapack (the configure script in ThirdParty/HSL definitely does), or
> whether the system libblas.a and liblapack.a on Debian were built with
> -fPIC.
>
> I don’t see why you only got static libraries out of running configure in
> ThirdParty/HSL. Can you post the config.log file from that?
>
> Otherwise I’d suggest you could build the Matlab mex interface yourself
> with all the HSL solvers compiled in. I recommend following the usage
> instructions in Ipopt/contrib/MatlabInterface/MatlabInterface.site, but
> there may be something strange about your system. Not sure.
>
> -Tony
>
>
>  *From:* Ipopt User <ipoptuser at gmail.com>
> *Sent:* Tuesday, September 30, 2014 8:41 AM
> *To:* ipopt at list.coin-or.org
> *Subject:* [Ipopt] compiling HSL loadable module under linux
>
>  I recently downloaded HSL for using better linear solvers in IPOPT.
> Unfortunately, I cannot get a loadable module to work.
>
> I am using Debian Wheezy, MATLAB R2013a x64 and the IPOPT3.11.8 Matlab
> interface (
> http://www.coin-or.org/download/binary/Ipopt/Ipopt-3.11.8-linux64mac64win32win64-matlabmexfiles.zip
> ), and wish to use the HSL shared library libhsl.so. Unfortunately, none of
> fancier linear solvers (MA77, MA87, MA97) works.
>
> First, I followed the directions from
> Ipopt-3.11.9/ThirdParty/HSL/INSTALL.HSL
> ~/Ipopt-3.11.9/ThirdParty/HSL$ ./configure --enable-loadable-library
> ADD_CXXFLAGS="-fPIC -fexceptions" ADD_CFLAGS="-fPIC -fexceptions"
> ADD_FFLAGS="-fPIC -fexceptions"
> ~/Ipopt-3.11.9/ThirdParty/HSL$ make
> it recognizes and compiles all HSL solvers, but the result of the
> compilation process is not a .so file:
>  :~/Ipopt-3.11.9/ThirdParty/HSL$ find . -regex '.*lib.*hsl.*'
> ./libcoinhsl.la
> ./coinhsl/libcoinhsl.sym
> ./.libs/libcoinhsl.a
> ./.libs/libcoinhsl.la
> ./.libs/libcoinhsl.lai
> ./lib/libcoinhsl.a
> ./lib/libcoinhsl.la
> ./lib/pkgconfig/coinhsl.pc
> ./lib/libhsl.so
> The last file is a symlink to a nonexisting file.
>
> Alternatively, I have tried to compile HSL directly. The steps I take to
> install the HSL shared library are:
> ~/coinhsl-2014.01.10$ ./configure
> ~/coinhsl-2014.01.10$ make
> ~/coinhsl-2014.01.10$ ln -s .libs/libcoinhsl.so .libs/libhsl.so
> ~/coinhsl-2014.01.10$ LD_LIBRARY_PATH="/home/user/coinhsl-2014.01.10"
> matlab
> These steps always succeed, and I can use MA27.
>
> When I use GCC 4.7.2 (ships with Debian Wheezy), I get error [1] (printed
> below this e-mail) whenever I try to use ipopt with one of the fancier HSL
> solvers in Matlab. This can be solved by running Matlab with a current
> version of libgfortran (see [2]). However, Matlab crashes on fancier
> solvers, e.g., MA87 gives [3], MA87 gives "*** glibc detected ***
> /usr/local/MATLAB/R2013a/bin/glnxa64/MATLAB: corrupted double-linked list:
> 0x000000000312d2a0 ***" and MA97 gives "MKL ERROR: Parameter 8 was
> incorrect on entry to DGEMM".
>
> When I use GCC 4.4.7 (which is the recommended version for compiling mex
> files for Matlab 2013a), error [1] does not appear. When I use the solver
> MA87, Matlab crashes as soon as it calls the solver [4]. When using MA77,
> IPOPT reports "WARNING: Problem in step computation; switching to emergency
> mode." MA97 gives "MKL ERROR: Parameter 3 was incorrect on entry to DGEMM".
> and exists with a message similar to [3] (the differences are in the
> hexagonal numbers).
>
> HSL has responded to this issue with:
> >> If MA27 works and nothing else does, it suggests that its probably
> related to the BLAS API. Mathworks, for reasons only know to themselves,
> decided to use the ILP64 interface rather than the standard LP64 interface
> [i.e. all integer parameters to BLAS calls need to be 64-bit not 32-bit].
> This causes a massive headache for anyone calling BLAS from a mex binary,
> and can lead to the sorts of problems you indicate as the pointers the BLAS
> think they've been passed are wrong.
>
> >> In terms of fixing it, the easiest way is probably to figure out how to
> link against an LP64 BLAS library for the mex binary - but be careful as
> Ipopt may be using ILP64 for its own calls to the BLAS if compiled with the
> matlab interface. A different trick is to compile the HSL codes with
> -fdefault-integer-8, but that may then break the ipopt/hsl interface
> instead.
>
> Their suggestion is to try this mailing list, since "(..) it may be that
> by compiling the HSL codes using the ipopt build system they've already
> incorporated a fix for the issue (rather than using the precompiled
> library)." However, as you can see at the start of this message, the
> standard method of compiling libhsl.so does not work.
>
> Any help would be greatly appreciated.
>
>
> ------------------------------
> _______________________________________________
> Ipopt mailing list
> Ipopt at list.coin-or.org
> http://list.coin-or.org/mailman/listinfo/ipopt
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://list.coin-or.org/pipermail/ipopt/attachments/20141001/396543d6/attachment-0001.html>