[Ipopt] Seeking to use HSL linear solvers with Ipopt, and call from R or Julia, on Windows 10 64-bit machine

Wed Jun 7 05:26:19 EDT 2017

I'd avoid using msys1, as it's obsolete, buggy, and unmaintained. Usually
msys2 or cygwin (using the mingw-w64 cross compilers) work fine for me, I
tend to prefer cygwin since it matches more closely with how you can also
cross compile windows dlls from Linux.

I haven't compiled Ipopt or HSL by hand on Windows in a long time, but
processes that worked a few years ago should all still work. I'd run
through some steps myself to give you a known working set of instructions,
probably based on the environment we use to build julia itself (see the
readme.windows.md description of which cygwin packages to select), but I
can't find any copies of coinhsl sources lying around on any of the
computers I use lately and am no longer academically affiliated (this email
address is about to stop working).

On Jun 7, 2017 2:19 AM, "Antonello Lobianco" <antonello at lobianco.org> wrote:

> I know it could look provocative, but if this is a large share of your
> workload, you could consider using Linux and compiling  IPOPT there..
> that's as simple as putting the HSL libs in the right place, call the Ipopt
> configure script (with `--prefix=/usr` in ubuntu to then make your compiled
> Ipopt systemwise available) and then just install Julia/JuMP or R...
> (I keep some slightly more detailed notes here:
> https://ffsm-project.org/wiki/en/dev/development#ipopt)
>
> Antonello
>
> On 5 June 2017 at 19:15, Don Boyd <donboyd5 at gmail.com> wrote:
>
>>
>> Hi all,
>>
>> I am spinning my wheels so much that I must ask for help. I know little
>> about configure...make... and think I am messing things up with
>> experimentation re path, compiler setting, compiler flags, etc. I don't
>> really understand these things very well.
>>
>> Goal: use HSL linear solvers with Ipopt, and call Ipopt from R or Julia
>> (just one will do), on Windows 10 64-bit machine (AMD 8-core FX-8350, 32gb
>> RAM)
>>
>> I am hoping you can recommend one or two approaches to accomplish this.
>> Details follow.
>>
>> I have academic license to HSL source. Have been unsuccessful at dozens
>> and dozens of attempts to either (1) compile HSL with Ipopt , or (2)
>> compile HSL as libhsl.dll (along with libmetis.dll, which libhsl.dll
>> needs) to use with Ipopt.
>>
>> Either of two alternative approaches would be great for me:
>>
>> 1. R:  Compile libhsl.dll (and libmetis.dll I think) in a manner that
>> works with my existing Ipoptr package interface. (I have had success using
>> Ipoptr with the pardiso dll, so I think this should work).
>>
>> Considerations: (a) in 2015 I compiled Ipoptr with Ipopt 3.12
>> successfully after dozens of attempts but have never been able to recompile
>> since, (b) I used (i) the Rtools 3.3 toolchain and (ii) R's Rblas.dll and
>> Rlapack.dll. I am assuming if I compile an HSL dll to load, I should build
>> it with the same (?) toolchain, and Rblas and Rlapack.
>>
>> 2. Julia: Compile libhsl.dll and libmetis.dll. (I have had success using
>> Ipopt.jl with the pardiso dll so I think this should work.) I assume I do
>> not have to compile with the Rtools toolchain or with Rblas.dll and
>> Rlapack.dll, but could. Also, I assume I would not have to compile Ipopt,
>> as Julia installs it so nicely in Ipopt.jl (that would be an enormous
>> relief).
>>
>> #2 might be best, as (a) Julia will be fast, and (b) it would force me to
>> learn more Julia.
>>
>> (Another option that would be fine is (3) full compilation of HSL with
>> Ipopt, for use with Ipoptr in R, or for use with Ipopt.jl in Julia, but
>> that seems harder since I fail at all attempts to compile Ipopt by itself.)
>>
>>
>> My attempts:
>>
>> I am not sure what information to provide because I have failed at
>> probably 50 or more variations.
>>
>> Let me describe my most recent failure to build libhsl.dll, since
>> building that might be the most desirable outcome. I can provide any
>> follow-up information that is needed.
>>
>> 1. Computer environment:
>>
>> 1.1 Compilers:
>> Rtools 3.3, freshly re installed today, as administrator, from
>> https://cran.rstudio.com/bin/windows/Rtools/Rtools34.exe, located in
>> C:\Rtools\mingw_64
>>
>> I have deleted all other compilers from my system to avoid confusion.
>> (Deleted separate MinGW installation, and separate TDM installation.)
>>
>> 1.2  Bash and its environment
>> - msys.bat and the MinGW tools, reinstalled today, as administrator:
>> installed mingw32-base, msys-base, and mingw-developer-toolkit. Did not
>> install compilers.
>> - edited fstab file to include 1 and only 1 substantive line:
>>       C:/Rtools/mingw_64 /mingw
>>
>> 1.3 Path
>>
>> Path=C:\Ipopt2017\Pardiso\libpardiso500-WIN-X86-64;C:\Program
>> Files\RStudio\bin\pandoc;C:\ProgramData\Oracle\Java\javapath
>> ;C:\PROGRA~1\R\R-34~1.0\bin\x64;C:\Program Files\R\R-3.4.0\bin;C:\WINDOWS
>> ;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\Program
>> Files (x86)\ATI Technologies\ATI.ACE\Core-Static;C:\Program
>> Files\EmEditor;C:\Strawberry\c\bin;C:\Strawberry\perl\site\b
>> in;C:\Strawberry\perl\bin;%SystemRoot%\system32;%SystemRoot%
>> ;%SystemRoot%\System32\Wbem;%SYSTEMROOT%\System32\
>> WindowsPowerShell\v1.0\;C:\ProgramData\chocolatey\bin;C:\Program
>> Files\Java\jdk1.7.0_79\bin;C:\Program Files\MiKTeX
>> 2.9\miktex\bin\x64\;C:\Program Files\TortoiseSVN\bin;C:\Program
>> Files\Mosek\8\tools\platform\win64x86\bin;c:\mingw\msys\1.0\bin
>>
>> 2. Sources:
>>
>> 2.1 HSL routines
>> - downloaded coinhsl-2014.01.10.zip
>> - extracted to its own directory per the HSL README (renamed it to
>> coinhsl but don't think rename needed)
>>
>> 2.2 Metis
>> - downloaded Metis 4.0.4 from http://glaros.dtc.umn.edu/gkho
>> me/fetch/sw/metis/OLD/metis-4.0.3.tar.gz and put in its own subdirectory
>> of coinhsl, named metis-4.0.3
>>
>> 3. Build environment
>> Created subdirectory ../coinhsl/build/x64 as location for the build
>>
>> 3. Configuration
>> 3.0 Opened Msys script shell as administrator
>>
>> 3.1 Preliminaries:
>> where make gives:
>> C:\MinGW\msys\1.0\bin\make.exe
>> (Not sure if I should want it to find the Rtools make instead)
>>
>> 3.2 Results of gcc -v
>> $ gcc -v
>> Using built-in specs.
>> COLLECT_GCC=C:\Rtools\mingw_64\bin\gcc.exe
>> COLLECT_LTO_WRAPPER=C:/Rtools/mingw_64/bin/../libexec/gcc/x8
>> 6_64-w64-mingw32/4.9.3/lto-wrapper.exe
>> Target: x86_64-w64-mingw32
>> Configured with: ../../../src/gcc-4.9.3/configure
>> --host=x86_64-w64-mingw32 --build=x86_64-w64-mingw32
>> --target=x86_64-w64-mingw32 --prefix=/mingw64 --with-sysroot=/home/Jeroen/mi
>> ngw-gcc-4.9.3/x86_64-493-posix-seh-rt_v3-s/mingw64
>> --with-gxx-include-dir=/mingw64/x86_64-w64-mingw32/include/c++
>> --enable-static --disable-shared --disable-multilib
>> --enable-languages=c,c++,fortran,lto --enable-libstdcxx-time=yes
>> --enable-threads=posix --enable-libgomp --enable-libatomic --enable-lto
>> --enable-graphite --enable-checking=release --enable-fully-dynamic-string
>> --enable-version-specific-runtime-libs --disable-isl-version-check
>> --disable-cloog-version-check --disable-libstdcxx-pch
>> --disable-libstdcxx-debug --enable-bootstrap --disable-rpath
>> --disable-win32-registry --disable-nls --disable-werror --disable-symvers
>> --with-gnu-as --with-gnu-ld --with-arch=nocona --with-tune=core2
>> --with-libiconv --with-system-zlib --with-gmp=/home/Jeroen/mingw-
>> gcc-4.9.3/prerequisites/x86_64-w64-mingw32-static
>> --with-mpfr=/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-w64-mingw32-static
>> --with-mpc=/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-w64-mingw32-static
>> --with-isl=/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-w64-mingw32-static
>> --with-cloog=/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-w64-mingw32-static
>> --enable-cloog-backend=isl --with-pkgversion='x86_64-posix-seh, Built by
>> MinGW-W64 project' --with-bugurl=http://sourcefor
>> ge.net/projects/mingw-w64 CFLAGS='-O2 -pipe
>> -I/home/Jeroen/mingw-gcc-4.9.3/x86_64-493-posix-seh-rt_v3-s/mingw64/opt/include
>> -I/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-zlib-static/include
>> -I/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-w64-mingw32-static/include'
>> CXXFLAGS='-O2 -pipe -I/home/Jeroen/mingw-gcc-4.9.3
>> /x86_64-493-posix-seh-rt_v3-s/mingw64/opt/include
>> -I/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-zlib-static/include
>> -I/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-w64-mingw32-static/include'
>> CPPFLAGS= LDFLAGS='-pipe -L/home/Jeroen/mingw-gcc-4.9.3
>> /x86_64-493-posix-seh-rt_v3-s/mingw64/opt/lib
>> -L/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-zlib-static/lib
>> -L/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-w64-mingw32-static/lib
>> '
>> Thread model: posix
>> gcc version 4.9.3 (x86_64-posix-seh, Built by MinGW-W64 project)
>>
>>
>> 3.3 Simple configuration from within ../coinhsl/build/x64
>> Configured with:
>> ../../configure --enable-shared --with-blas="-LC:/PROGRA~1/R/R-34~1.0/bin/x64
>> -lRblas"
>>
>> To my inexpert eyes, it looks successful: config.log had exit 0. But
>> there seemed to be problems finding some lapack functions but I don't
>> really understand it or know what to look for.
>>
>> 4. Make
>> 4.1 The make seemed to start out right but then it had a lot of undefined
>> references related to fortran code and functions that I gather are part of
>> lapack. Here are the first few lines in the make process that seem to
>> indicate problems:
>>
>> libtool: link: if test "x`/bin/sed 1q .libs/libcoinhsl.def`" = xEXPORTS;
>> then cp .libs/libcoinhsl.def .libs/libcoinhsl-0.dll.def; else echo EXPORTS
>> > .libs/libcoinhsl-0.dll.def; cat .libs/libcoinhsl.def >>
>> .libs/libcoinhsl-0.dll.def; fi
>> libtool: link:  gfortran -shared .libs/libcoinhsl-0.dll.def
>>  common/.libs/deps90.o common/.libs/deps.o common/.libs/dump.o
>> mc19/.libs/mc19d.o ma27/.libs/ma27d.o ma28/.libs/ma28d.o ma57/.libs/ma57d.o
>> hsl_ma77/.libs/hsl_ma77d.o hsl_ma77/C/.libs/hsl_ma77d_ciface.o
>> hsl_ma86/.libs/hsl_ma86d.o hsl_ma86/C/.libs/hsl_ma86d_ciface.o
>> hsl_mc68/C/.libs/hsl_mc68i_ciface.o hsl_ma97/.libs/hsl_ma97d.o
>> hsl_ma97/C/.libs/hsl_ma97d_ciface.o loadmetis/.libs/loadmetis.o    -O2
>> -o .libs/libcoinhsl-0.dll -Wl,--enable-auto-image-base -Xlinker
>> --out-implib -Xlinker .libs/libcoinhsl.dll.a
>> common/.libs/deps90.o: In function `update':
>> c:\Ipopt2017\hslnew\coinhsl\build\x64/../../common/deps90.f90:2402:
>> undefined reference to `dgemm_'
>> common/.libs/deps90.o: In function `update2':
>> c:\Ipopt2017\hslnew\coinhsl\build\x64/../../common/deps90.f90:2456:
>> undefined reference to `dcopy_'
>> c:\Ipopt2017\hslnew\coinhsl\build\x64/../../common/deps90.f90:2463:
>> undefined reference to `dgemm_'
>>
>> 4.2 And here are the last few lines of the make process:
>> c:\Ipopt2017\hslnew\coinhsl\build\x64/../../hsl_ma97/hsl_ma97d.f90:5900:
>> undefined reference to `dtrsm_'
>> hsl_ma97/.libs/hsl_ma97d.o: In function `__hsl_ma97_double_MOD_ma97_so
>> lve_fredholm_double':
>> c:\Ipopt2017\hslnew\coinhsl\build\x64/../../hsl_ma97/hsl_ma97d.f90:4487:
>> undefined reference to `dnrm2_'
>> collect2.exe: error: ld returned 1 exit status
>> make[1]: *** [libcoinhsl.la] Error 1
>> make[1]: Leaving directory `/c/Ipopt2017/hslnew/coinhsl/build/x64'
>> make: *** [all] Error 2
>>
>>
>> So that's about it. I can provide config.log, etc., if at all useful. All
>> other things I have tried have resulted in errors, too, but often different
>> kinds of errors. I would much appreciate advice.
>>
>> Again, I would be very happy to have this work under Julia or under R, in
>> Windows 64-bit - an HSL dll would be ideal, but full compilation of HSL
>> with Ipopt also would be fine.
>>
>> Thank you very much in advance.
>>
>> Don Boyd
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> Ipopt mailing list
>> Ipopt at list.coin-or.org
>> https://list.coin-or.org/mailman/listinfo/ipopt
>>
>>
>
>
> --
> Antonello Lobianco
> AgroParisTech, Laboratoire d'Economie Forestière
> 14 Rue Girardet - 54000 Nancy, France
> Tel: +33.383396865 <+33%203%2083%2039%2068%2065>
> Email, university: antonello.lobianco at inra.fr
> <antonello.lobianco at nancy.inra.fr>
> Email, personal: antonello at lobianco.org
> http://antonello.lobianco.org
>
> _______________________________________________
> Ipopt mailing list
> Ipopt at list.coin-or.org
> https://list.coin-or.org/mailman/listinfo/ipopt
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://list.coin-or.org/pipermail/ipopt/attachments/20170607/720d331b/attachment-0001.html>