[Ipopt] Numerical differences between runs
Andreas Waechter
andreasw at watson.ibm.com
Fri Feb 26 15:58:16 EST 2010
Hi Gwyn,
In principle, the Ipopt code is deterministic, and results should be 100%
reproducible. This assumes that we are talking about running the same
problem with exactly the same binary input repeatedly.
Possible causes for non-repeatability that I can imagine are:
- A memory bug (in your code or Ipopt code). If uninitialized memory is
used, the result is unpredictable. Can you run your code through a memory
checker?
- What linear solver are you using? And what BLAS are you using? I'm not
sure what is in the Windows DLL binaries, but my guess is that it is MUMPS
and netlib BLAS (I'm not compiling the DLLs, and I forgot what Marcel puts
in there). This should be deterministic code. But maybe you are using a
different linear solver, or some different BLAS (such as the MKL)? When
you compare the output with print level J_MOREVECTOR, are the first
differences observed in the solution of the linear solver (you write resid
and rhs, but which one is the first, or is it in a "SOL" component)? If
so, it probably depends on what linear solver (and underlying BLAS) you
are using to see how that could be made reproducible. Also, I assume that
you are not running things in parallel/multi-threaded.
- I assume that you are not using any Ipopt warmstart (that would not
guarantee repeatability). And are you using always the same
IpoptApplication, or a new one every time? Maybe the non-repeatability
goes away if you always use a new IpoptApplication? (If that is the case,
it means that Ipopt carries some unintended information from one solve to
the next, and I would like to know about that).
In general, for difficult problems, I have often observed a different
number of iterations (or sometimes success vs. failure) when compiling the
code with different compiler options, or when changing the operating
system, particularly for difficult problems. Unfortunately, it is in the
nature of the floating point operations that the solution of the linear
systems might differ drastically if those linear systems are
ill-conditioned, and some floating point operations are done in a
different order (which may be mathematically equivalent in precise
arithmetic).
Not sure if this helps, but it might point you into the right direction.
Regards,
Andreas
On Wed, 24 Feb 2010, Gwyn Roberts wrote:
> We are relatively new users of IpOpt and have been gaining
experience over the last several months with a small to medium scale (2-4K
variables/equations, 8-10K NZ (50% nonlinear)) nonlinear model
application. We are using the version 3.8.1 release binariesfor Windows
(as a DLL). Our full scale models are to the point where they solve quite
nicely and robustly for most cases. However, we are encountering
unexplained differences in â€œtrajectoryâ€ (e.g. number of iterations,
final convergence tightness, etc.) from run to run. This is the nature of
our main concern â€“ repeatability of performance and results. For our
full model runs, the differences are manifest as different behaviours that
range from 1 or 2 extra iterations to some that near the point of closure
end up tripling the number of iterations and therefore the times.
We find that the trajectory for a problem will change depending on the
problem solved previously, or, if running only a single problem, will
change depending on whether or not a file journal is added. From looking
at J_MOREVECTOR level output, we see that the departure in trajectory
seems to take a while to take hold (3-8 iterations). The first observable
behaviour is around the Barrier Parameter Update step and the max norm
calculations related to the underlying calculations there (we have settled
on adaptive strategy, but see the issues with monotone as well). There are
numerical differences in the 8-15th significant figures of the resid and
rhs vectors displayed at those journal levels. These appear ultimately to
affect the relevant norms in significant ways that may alter the behaviour
of the barrier slightly.
Can you comment on whether this is behaviour you have seen or experienced
yourself (i.e. this is known behaviour, issue)? If you have any thoughts
or guidance (or code changes) that might help us to eliminate the floating
point variability, we would greatly appreciate your help. We have not yet
taken the step of doing a full build of IpOpt ourselves, but have the
ability to do so. We would wait to hear your reaction and thoughts before
taking that on.
Thank you for your time and assistance.
Gwyn
More information about the Ipopt
mailing list