[Ipopt] Hessian Computation
Andreas Waechter
andreasw at watson.ibm.com
Tue Apr 20 12:39:07 EDT 2010
Hi Chuck,
(Sorry for the late reply, as you can see, I'm working my way backwards
through the Ipopt emails...)
I'm not sure I understand, so this is what I understood: In AIMMS, you
can compute second derivatives for "internal" function, but not for
external functions, so you simple ignore the second derivatives for
external functions. I'm surprised this works as well as you describe, but
I guess if this is only a subset you might be getting away with it well.
I'm a little surprised you write that Ipopt takes more time per iteration
when you use the LBFGS Hessian approximation, since then the matrix that
is factorized is even easier than in the case where you ignore parts of
the derivatives (or at least it is roughly the same as if the exact
Hessian is only a diagonal). However, depending on how large you set the
limited_memory_max_history option, a number of backsolves have to be
performed, so, you might get faster times per iteraion if you decrease
this option, but then the result might be that overall you need more
iterations...
>From a theoretical point of view, in order to guarantee global
convergence, you can essentially provide any (uniformly bounded) matrix as
Hessian matrix, since Ipopt will make sure it is positive definite in the
null space of the constraints (well, there are more assumptions to prove
global convergence, but simplified speaking, the is very little required
for the Hessian). But of course the closer the matrix is to the true
Hessian, the faster convergence will usually be.
In any case, I probably didn't bring much light into this issue; but it
was interesting to hear your observation that ignoring some of the second
derivatives still works pretty well!
Regards,
Andreas
On Sun, 4 Apr 2010, Chuck Teeter wrote:
> Hi All,
>
> I have a question about the options available for Hessian computation in
> Ipopt. First, some background: I use Ipopt with the AIMMS modeling system
> for refinery planning. My model makes use of some external, nonlinear
> constraint functions which are implemented in C++. For these functions, I
> also use automatic differentiation to provide first partial derivatives to
> AIMMS (and hence on to Ipopt). It just so happens that in AIMMS, if
> external functions are used as model constraints, second derivatives are not
> made available to Ipopt for ANY constraints, either constraints implemented
> natively in AIMMS, or constraints that are implemented in external
> functions.
>
> When solving my model, I set the method for Hessian computation to "exact".
> It solves quickly and reliably 99% of the time without ANY Hessian
> information from AIMMS. For comparison purposes, I have also used the
> quasi-newton Hessian approximation option. It also solves reliably as well,
> although it takes much longer per iteration because it now has to do much
> more work. I found these results to be somewhat curious, in that solution
> reliability does not appear to be adversely affected by lack of Hessian
> information or lack of Hessian quality. I have a variety of nonlinear
> constraints in my model. Although some nonlinear constraints involve
> bilinear terms, most are far more complex and are definitely 2nd order
> differentiable.
>
> I guess my questions boil down to these: Is this result expected? Would
> you expect significantly improved performance/reliability if exact Hessian
> information was made available to Ipopt? Under what conditions would you
> expect accurate Hessian information to be critical in achieving quick and/or
> reliable solutions?
>
> Thanks in advance for any insights!
>
> Chuck
>
More information about the Ipopt
mailing list