[Ipopt] continously run IPOPT,but different results

[Tony Kelman]

Additionally, are you absolutely sure you're not doing any type of warm 
starting, and every single input to the Ipopt call is exactly the same each 
time?

The constraint dependency detector is not a default setting and I believe 
it's described as experimental, so unless Felix is specifically trying to 
use it I doubt that's the source of the issue.

I've seen 3 causes of non-deterministic behavior in the Ipopt Matlab 
interface:

1. Accumulated roundoff errors from parallel BLAS/LAPACK libraries (applies 
for other interfaces to Ipopt too). These should really only show up in the 
last few decimal places, unless the problem has multiple non-unique 
solutions or is poorly conditioned. For non-convex problems, this could even 
lead Ipopt to converge to different local minima - if the exit flag is 
always solve succeeded but the optimal objective values differ 
significantly, this may be what's happening.
2. Mistakenly changing the initial point, warm starting the primal and/or 
dual variables with different values each time. Review your code carefully, 
you might be reusing some structures that you thought were constant but 
actually changing certain fields, etc. This might be less likely when you 
call Ipopt through OPTI, I'm not positive.
3. Memory access bugs in the Matlab interface. I believe I've found and 
fixed all of these and if you're using the latest version of OPTI then 
Jonathan has implemented similar fixes.

If reviewing your Matlab code doesn't show anything (and Jonathan's the best 
person to look over how you call OPTI), it's best to increase the print 
level (using output_file and file_print_level to avoid filling your command 
window) and compare detailed outputs from different calls to see exactly 
where the results begin to differ. If the cause is #1, you can try switching 
to a mex file that was compiled using the reference versions of BLAS/LAPACK 
(e.g. 
http://www.coin-or.org/download/binary/Ipopt/Ipopt-trunk2112-linux64mac64win32win64-matlabmexfiles.zip).

-Tony

-----Original Message----- 
From: Jonathan Currie <jonathan.currie at aut.ac.nz>
Date: Thu, 18 Apr 2013 18:27:24 +1200
To: "'cat fa'" <boost.subscribing at gmail.com>, <ipopt at list.coin-or.org>
Subject: Re: [Ipopt] continously run IPOPT,but different results

Hi Felix,

>From my understanding of the IPOPT algorithm it should return exactly the
same solution for each run of an identical problem, assuming infinite
precision. I see IPOPT does use some random numbers when testing for
dependent equality constraints, perhaps someone could comment on whether
this could cause non-deterministic behaviour?

A couple of important questions for your problem:
1) By slightly different results, how many decimal places are you referring
to?
2) Is your problem very nearly infeasible?
3) Is there a significant dynamic range in your problem (i.e. large range of
numbers)?

Both points 2 and 3 can result in poor numerical stability, which combined
with non-deterministic linear algebra libraries (due to threading, memory
alignment - both inferring a pseudo-random order of floating point
operations), can lead to noticeable differences in the operation of the
algorithm. I have seen this behaviour in CSDP and found it incredibly
difficult to isolate. For more information see the following article:

http://software.intel.com/en-us/articles/getting-reproducible-results-with-i
ntel-mkl

Remember I compiled the OPTI version of IPOPT against a multi-threaded build
of Intel MKL. From the user's point of view, there is not much you can do to
solve this using OPTI. You will need to compile IPOPT yourself using a
sequential build of BLAS/LAPACK and ensure memory alignment does not cause
non-deterministic behaviour, in order to see whether this is causing your
problem. If you can send me your model and associated code I am happy to
have a look for you.

Also, perhaps somebody else on this forum can offer some further ideas on
this problem.

Jonathan Currie
OPTI Developer