[CppAD] Lagrangian Hessians

[Brad Bell]

Kipp asked two interesting and related questions. One about constants 
and variables and the other about re-taping calculations. You can view 
my answer in the form of a new CppAD example. See
   http://www.coin-or.org/CppAD/Doc/heslagrangian.cpp.xml
(use the *.htm extension if you browser does not support MathML.)


Kipp Martin wrote:

> Hi Brad:
>
> I am in the process of coding the nonlinear API for our OS project 
> that uses CppAD.
>
> Some nonlinear solvers that use second derivative information (e.g. 
> the Hessian) want the Hessian of the Lagrangian function. That is, you 
> multiply all the constraints by a vector of Lagrange multipliers, add 
> to the objective function, and return the Hessian of that to the 
> solver. The vector of Lagrange multipliers may change at each 
> iteration. One way to do this is to calculate the Hessian of each 
> constraint using CppAD, multiply the result by the appropriate 
> Lagrange multiplier, and then add the resulting Hessian matrices.  A 
> second approach is to form the Lagragian function and then have CppAD 
> take the Hessian of the Lagragian function. With the second approach 
> the Hessian calculation now has the Lagrange multipliers in it and we 
> have increased the dimension of the problem. Is there an easy way to 
> tell CppAD NOT to take the derivative with respect to the Lagrange 
> multipliers? I would still like treat them as variables at each 
> iteration so they can change and I do not have to rebuild an 
> expression tree, but I don't want to take the derivative with respect 
> to these variables. Hopefully, this was clear.
>
> Thanks,

>
>Hi Brad:
>
>I think I have the Hessian calculation working inside our API. However,
> I would like to be able to calculate the Lagrangian as efficiently as
>possible at each iteration. In order to do so, I would like to do the
>"tape recording" of the constraints and objective function for the
>independent variables only ONCE. That is, declare my independent
>variables X and build f(X), g1(X), g2(X) , etc. only once, NOT at each
>iteration (this is what I do for the gradient calculations). Then AT
>EVERY iteration make the calculation
>
>z*F(X) + y1*g1(X) + ... + ym*gm(X)
>
>with the new values for X and Y and then take the Hessian with respect to X.
>
>However, there seems to be a problem. At the new iteration I can change
>the values of the independent variables and have it affect the
>calculation of the Hessian. However, the Hessian calculation is not
>affected by the new values of the y variables.
>
>Is there a way to take the updated (X, Y) variables at each iteration,
>use the stored values for F(X), g1(X), etc. then make the calculation
>
>z*F(X) + y1*g1(X) + ... + ym*gm(X)
>
>and calculate the Hessian? This is not a problem for the gradient I only
>have independent variables and can use the stored values of F(X), g1(X)
>etc. at each iteration.
>
>Thanks
>
>