[Ipopt] Hessian Computation

[Andreas Waechter]

Hi Chuck,

(Sorry for the late reply, as you can see, I'm working my way backwards 
through the Ipopt emails...)

I'm not sure I understand, so this is what I understood:  In AIMMS, you 
can compute second derivatives for "internal" function, but not for 
external functions, so you simple ignore the second derivatives for 
external functions.  I'm surprised this works as well as you describe, but 
I guess if this is only a subset you might be getting away with it well. 
I'm a little surprised you write that Ipopt takes more time per iteration 
when you use the LBFGS Hessian approximation, since then the matrix that 
is factorized is even easier than in the case where you ignore parts of 
the derivatives (or at least it is roughly the same as if the exact 
Hessian is only a diagonal).  However, depending on how large you set the 
limited_memory_max_history option, a number of backsolves have to be 
performed, so, you might get faster times per iteraion if you decrease 
this option, but then the result might be that overall you need more 
iterations...

>From a theoretical point of view, in order to guarantee global 
convergence, you can essentially provide any (uniformly bounded) matrix as 
Hessian matrix, since Ipopt will make sure it is positive definite in the 
null space of the constraints (well, there are more assumptions to prove 
global convergence, but simplified speaking, the is very little required 
for the Hessian).  But of course the closer the matrix is to the true 
Hessian, the faster convergence will usually be.

In any case, I probably didn't bring much light into this issue; but it 
was interesting to hear your observation that ignoring some of the second 
derivatives still works pretty well!

Regards,

Andreas

On Sun, 4 Apr 2010, Chuck Teeter wrote:

> Hi All,
>
> I have a question about the options available for Hessian computation in
> Ipopt.  First, some background:  I use Ipopt with the AIMMS modeling system
> for refinery planning.  My model makes use of some external, nonlinear
> constraint functions which are implemented in C++.  For these functions, I
> also use automatic differentiation to provide first partial derivatives to
> AIMMS (and hence on to Ipopt).  It just so happens that in AIMMS, if
> external functions are used as model constraints, second derivatives are not
> made available to Ipopt for ANY constraints, either constraints implemented
> natively in AIMMS, or constraints that are implemented in external
> functions.
>
> When solving my model, I set the method for Hessian computation to "exact".
> It solves quickly and reliably 99% of the time without ANY Hessian
> information from AIMMS.  For comparison purposes, I have also used the
> quasi-newton Hessian approximation option.  It also solves reliably as well,
> although it takes much longer per iteration because it now has to do much
> more work.  I found these results to be somewhat curious, in that solution
> reliability does not appear to be adversely affected by lack of Hessian
> information or lack of Hessian quality.  I have a variety of nonlinear
> constraints in my model.  Although some nonlinear constraints involve
> bilinear terms, most are far more complex and are definitely 2nd order
> differentiable.
>
> I guess my questions boil down to these:  Is this result expected?  Would
> you expect significantly improved performance/reliability if exact Hessian
> information was made available to Ipopt?  Under what conditions would you
> expect accurate Hessian information to be critical in achieving quick and/or
> reliable solutions?
>
> Thanks in advance for any insights!
>
> Chuck
>