[Ipopt] Seeking to use HSL linear solvers with Ipopt, and call from R or Julia, on Windows 10 64-bit machine

Wed Jun 7 04:56:09 EDT 2017

Hi,

building DLLs with autoconf, especially the old version that COIN-OR is 
still using, can be difficult to impossible. But from what you wrote, 
I'm not sure which configure you are actually using.

The output in 4. shows that Blas was not picked up.
You could dry configure in 3. with 
LDFLAGS="-LC:/PROGRA~1/R/R-34~1.0/bin/x64 -lRblas" instead of using 
--with-blas.

But as HSL is only a few files, just writing your own little script that 
calls the compiler and linker directly might also be an option. That 
would give you direct control about what all the flags.

Stefan

On 06/05/2017 07:15 PM, Don Boyd wrote:
> Hi all,
> 
> I am spinning my wheels so much that I must ask for help. I know little
> about configure...make... and think I am messing things up with
> experimentation re path, compiler setting, compiler flags, etc. I don't
> really understand these things very well.
> 
> Goal: use HSL linear solvers with Ipopt, and call Ipopt from R or Julia
> (just one will do), on Windows 10 64-bit machine (AMD 8-core FX-8350, 32gb
> RAM)
> 
> I am hoping you can recommend one or two approaches to accomplish this.
> Details follow.
> 
> I have academic license to HSL source. Have been unsuccessful at dozens and
> dozens of attempts to either (1) compile HSL with Ipopt , or (2) compile
> HSL as libhsl.dll (along with libmetis.dll, which libhsl.dll needs) to use
> with Ipopt.
> 
> Either of two alternative approaches would be great for me:
> 
> 1. R:  Compile libhsl.dll (and libmetis.dll I think) in a manner that works
> with my existing Ipoptr package interface. (I have had success using Ipoptr
> with the pardiso dll, so I think this should work).
> 
> Considerations: (a) in 2015 I compiled Ipoptr with Ipopt 3.12 successfully
> after dozens of attempts but have never been able to recompile since, (b) I
> used (i) the Rtools 3.3 toolchain and (ii) R's Rblas.dll and Rlapack.dll. I
> am assuming if I compile an HSL dll to load, I should build it with the
> same (?) toolchain, and Rblas and Rlapack.
> 
> 2. Julia: Compile libhsl.dll and libmetis.dll. (I have had success using
> Ipopt.jl with the pardiso dll so I think this should work.) I assume I do
> not have to compile with the Rtools toolchain or with Rblas.dll and
> Rlapack.dll, but could. Also, I assume I would not have to compile Ipopt,
> as Julia installs it so nicely in Ipopt.jl (that would be an enormous
> relief).
> 
> #2 might be best, as (a) Julia will be fast, and (b) it would force me to
> learn more Julia.
> 
> (Another option that would be fine is (3) full compilation of HSL with
> Ipopt, for use with Ipoptr in R, or for use with Ipopt.jl in Julia, but
> that seems harder since I fail at all attempts to compile Ipopt by itself.)
> 
> 
> My attempts:
> 
> I am not sure what information to provide because I have failed at probably
> 50 or more variations.
> 
> Let me describe my most recent failure to build libhsl.dll, since building
> that might be the most desirable outcome. I can provide any follow-up
> information that is needed.
> 
> 1. Computer environment:
> 
> 1.1 Compilers:
> Rtools 3.3, freshly re installed today, as administrator, from
> https://cran.rstudio.com/bin/windows/Rtools/Rtools34.exe, located in
> C:\Rtools\mingw_64
> 
> I have deleted all other compilers from my system to avoid confusion.
> (Deleted separate MinGW installation, and separate TDM installation.)
> 
> 1.2  Bash and its environment
> - msys.bat and the MinGW tools, reinstalled today, as administrator:
> installed mingw32-base, msys-base, and mingw-developer-toolkit. Did not
> install compilers.
> - edited fstab file to include 1 and only 1 substantive line:
>        C:/Rtools/mingw_64 /mingw
> 
> 1.3 Path
> 
> Path=C:\Ipopt2017\Pardiso\libpardiso500-WIN-X86-64;C:\Program
> Files\RStudio\bin\pandoc;C:\ProgramData\Oracle\Java\javapath;C:\PROGRA~1\R\R-34~1.0\bin\x64;C:\Program
> Files\R\R-3.4.0\bin;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\Program
> Files (x86)\ATI Technologies\ATI.ACE\Core-Static;C:\Program
> Files\EmEditor;C:\Strawberry\c\bin;C:\Strawberry\perl\site\bin;C:\Strawberry\perl\bin;%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;%SYSTEMROOT%\System32\WindowsPowerShell\v1.0\;C:\ProgramData\chocolatey\bin;C:\Program
> Files\Java\jdk1.7.0_79\bin;C:\Program Files\MiKTeX
> 2.9\miktex\bin\x64\;C:\Program Files\TortoiseSVN\bin;C:\Program
> Files\Mosek\8\tools\platform\win64x86\bin;c:\mingw\msys\1.0\bin
> 
> 2. Sources:
> 
> 2.1 HSL routines
> - downloaded coinhsl-2014.01.10.zip
> - extracted to its own directory per the HSL README (renamed it to coinhsl
> but don't think rename needed)
> 
> 2.2 Metis
> - downloaded Metis 4.0.4 from
> http://glaros.dtc.umn.edu/gkhome/fetch/sw/metis/OLD/metis-4.0.3.tar.gz and
> put in its own subdirectory of coinhsl, named metis-4.0.3
> 
> 3. Build environment
> Created subdirectory ../coinhsl/build/x64 as location for the build
> 
> 3. Configuration
> 3.0 Opened Msys script shell as administrator
> 
> 3.1 Preliminaries:
> where make gives:
> C:\MinGW\msys\1.0\bin\make.exe
> (Not sure if I should want it to find the Rtools make instead)
> 
> 3.2 Results of gcc -v
> $ gcc -v
> Using built-in specs.
> COLLECT_GCC=C:\Rtools\mingw_64\bin\gcc.exe
> COLLECT_LTO_WRAPPER=C:/Rtools/mingw_64/bin/../libexec/gcc/x86_64-w64-mingw32/4.9.3/lto-wrapper.exe
> Target: x86_64-w64-mingw32
> Configured with: ../../../src/gcc-4.9.3/configure --host=x86_64-w64-mingw32
> --build=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --prefix=/mingw64
> --with-sysroot=/home/Jeroen/mingw-gcc-4.9.3/x86_64-493-posix-seh-rt_v3-s/mingw64
> --with-gxx-include-dir=/mingw64/x86_64-w64-mingw32/include/c++
> --enable-static --disable-shared --disable-multilib
> --enable-languages=c,c++,fortran,lto --enable-libstdcxx-time=yes
> --enable-threads=posix --enable-libgomp --enable-libatomic --enable-lto
> --enable-graphite --enable-checking=release --enable-fully-dynamic-string
> --enable-version-specific-runtime-libs --disable-isl-version-check
> --disable-cloog-version-check --disable-libstdcxx-pch
> --disable-libstdcxx-debug --enable-bootstrap --disable-rpath
> --disable-win32-registry --disable-nls --disable-werror --disable-symvers
> --with-gnu-as --with-gnu-ld --with-arch=nocona --with-tune=core2
> --with-libiconv --with-system-zlib
> --with-gmp=/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-w64-mingw32-static
> --with-mpfr=/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-w64-mingw32-static
> --with-mpc=/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-w64-mingw32-static
> --with-isl=/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-w64-mingw32-static
> --with-cloog=/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-w64-mingw32-static
> --enable-cloog-backend=isl --with-pkgversion='x86_64-posix-seh, Built by
> MinGW-W64 project' --with-bugurl=http://sourceforge.net/projects/mingw-w64
> CFLAGS='-O2 -pipe
> -I/home/Jeroen/mingw-gcc-4.9.3/x86_64-493-posix-seh-rt_v3-s/mingw64/opt/include
> -I/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-zlib-static/include
> -I/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-w64-mingw32-static/include'
> CXXFLAGS='-O2 -pipe
> -I/home/Jeroen/mingw-gcc-4.9.3/x86_64-493-posix-seh-rt_v3-s/mingw64/opt/include
> -I/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-zlib-static/include
> -I/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-w64-mingw32-static/include'
> CPPFLAGS= LDFLAGS='-pipe
> -L/home/Jeroen/mingw-gcc-4.9.3/x86_64-493-posix-seh-rt_v3-s/mingw64/opt/lib
> -L/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-zlib-static/lib
> -L/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-w64-mingw32-static/lib '
> Thread model: posix
> gcc version 4.9.3 (x86_64-posix-seh, Built by MinGW-W64 project)
> 
> 
> 3.3 Simple configuration from within ../coinhsl/build/x64
> Configured with:
> ../../configure --enable-shared
> --with-blas="-LC:/PROGRA~1/R/R-34~1.0/bin/x64 -lRblas"
> 
> To my inexpert eyes, it looks successful: config.log had exit 0. But there
> seemed to be problems finding some lapack functions but I don't really
> understand it or know what to look for.
> 
> 4. Make
> 4.1 The make seemed to start out right but then it had a lot of undefined
> references related to fortran code and functions that I gather are part of
> lapack. Here are the first few lines in the make process that seem to
> indicate problems:
> 
> libtool: link: if test "x`/bin/sed 1q .libs/libcoinhsl.def`" = xEXPORTS;
> then cp .libs/libcoinhsl.def .libs/libcoinhsl-0.dll.def; else echo EXPORTS
>> .libs/libcoinhsl-0.dll.def; cat .libs/libcoinhsl.def >>
> .libs/libcoinhsl-0.dll.def; fi
> libtool: link:  gfortran -shared .libs/libcoinhsl-0.dll.def
>   common/.libs/deps90.o common/.libs/deps.o common/.libs/dump.o
> mc19/.libs/mc19d.o ma27/.libs/ma27d.o ma28/.libs/ma28d.o ma57/.libs/ma57d.o
> hsl_ma77/.libs/hsl_ma77d.o hsl_ma77/C/.libs/hsl_ma77d_ciface.o
> hsl_ma86/.libs/hsl_ma86d.o hsl_ma86/C/.libs/hsl_ma86d_ciface.o
> hsl_mc68/C/.libs/hsl_mc68i_ciface.o hsl_ma97/.libs/hsl_ma97d.o
> hsl_ma97/C/.libs/hsl_ma97d_ciface.o loadmetis/.libs/loadmetis.o    -O2   -o
> .libs/libcoinhsl-0.dll -Wl,--enable-auto-image-base -Xlinker --out-implib
> -Xlinker .libs/libcoinhsl.dll.a
> common/.libs/deps90.o: In function `update':
> c:\Ipopt2017\hslnew\coinhsl\build\x64/../../common/deps90.f90:2402:
> undefined reference to `dgemm_'
> common/.libs/deps90.o: In function `update2':
> c:\Ipopt2017\hslnew\coinhsl\build\x64/../../common/deps90.f90:2456:
> undefined reference to `dcopy_'
> c:\Ipopt2017\hslnew\coinhsl\build\x64/../../common/deps90.f90:2463:
> undefined reference to `dgemm_'
> 
> 4.2 And here are the last few lines of the make process:
> c:\Ipopt2017\hslnew\coinhsl\build\x64/../../hsl_ma97/hsl_ma97d.f90:5900:
> undefined reference to `dtrsm_'
> hsl_ma97/.libs/hsl_ma97d.o: In function
> `__hsl_ma97_double_MOD_ma97_solve_fredholm_double':
> c:\Ipopt2017\hslnew\coinhsl\build\x64/../../hsl_ma97/hsl_ma97d.f90:4487:
> undefined reference to `dnrm2_'
> collect2.exe: error: ld returned 1 exit status
> make[1]: *** [libcoinhsl.la] Error 1
> make[1]: Leaving directory `/c/Ipopt2017/hslnew/coinhsl/build/x64'
> make: *** [all] Error 2
> 
> 
> So that's about it. I can provide config.log, etc., if at all useful. All
> other things I have tried have resulted in errors, too, but often different
> kinds of errors. I would much appreciate advice.
> 
> Again, I would be very happy to have this work under Julia or under R, in
> Windows 64-bit - an HSL dll would be ideal, but full compilation of HSL
> with Ipopt also would be fine.
> 
> Thank you very much in advance.
> 
> Don Boyd
> 
> 
> 
> _______________________________________________
> Ipopt mailing list
> Ipopt at list.coin-or.org
> https://list.coin-or.org/mailman/listinfo/ipopt
> 

-- 
http://www.gams.com/~stefan