[CppAD] Diagonal of Hessian

[Brad Bell]

On Thu, 29 Apr 2010, xxxxxxx wrote:

> Dear Dr. Bell,
>
> thank you for answering my question so quickly. I will try your
> suggestions today.
>
> Just to be sure what supplying a sparsity pattern means: does it mean to
> tell cppAD which entries of the Hessian to compute? In the case of the
> diagonal, that would mean to pass the identity matrix to SparseHessian().


No. A sparsity pattern informs CppAD of which entires of a Hessian are 
known to be zero. For example, suppose that our function is
 	f(x1, x2) = x1^2 + x2^2 
then we know the off diagonal elements of the Hessian are zero. See the 
headings Vector of Boolean and Vector of Sets under Sparsity Pattern at
http://www.coin-or.org/CppAD/Doc/glossary.xml#Sparsity%20Pattern


>
> The physical background is that we want to try out several forms of
> electron-electron interactions and optimize parameters. This entails two
> steps, the first going into the Schrödinger equation with one's
> function, which does only contain derivatives of second order of the
> same variable, but no mixed derivatives. In the second step, one would
> of course need the complete Jacobian and Hessian.
>

Does the operation sequence for this function depend on the independent 
variable values. For example, the operation sequence for
 	f(x1, x2) = x1^2 + x2^2
does not depend on the value of (x1, x2). On the other hand
 	f(x1, x2) = x1^2 + x2^2   if x1 < 1
 	          = |x1| + x2^2   if x1 >= 1
does depend on the value of (x1, x2). See the heading Depend under 
Operation at
http://www.coin-or.org/CppAD/Doc/glossary.xml#Operation

If the operation sequence does not depend on the value of (x1, x2), you do 
not have to `retape' it for each argument value. You can find a 
comparision between the code (and speed) for retaping and not retaping at
http://www.coin-or.org/CppAD/Doc/cppad_poly.cpp.xml


> As I am now concerned with step one, efficient computation of the
> diagonal of the Hessian is the issue. Our current approach is to simply
> code and differentiate our functions by hand, an effort that is very
> time consuming and puts severe limits on the number of functions we can
> test. However, the resulting code is quite fast.
>
> Even so, it would be very useful to have an AD alternative, because one
> could test an approach on small systems and would then be able to decide
> whether it's worth to hand code a particular approach. In any case, I
> feel exploring AD methods would yield some reward either way. After
> trying operator overloading, I am quite excited about the fact that one
> gets the derivative without having to modify the original source coude.
>
> I'll let you know how the testing goes after I've gotten around
> implementing it! Thank you so much, also for providing cppAD in the
> first place!
>

As it turns out, my initial idea has some problems with it. I am going to 
be on the Philippine sea for work for the next month and do not think that 
I will get to working on this in the near future. But I will be able to 
log into e-mail and provide suggestions.


> Yours sincerely,
> xxxxxxx
>
>
> Brad Bell schrieb:
>>
>>
>>
>>
>> On Wed, 28 Apr 2010, xxxxxxx wrote:
>>
>>> Dear Dr. Bell,
>>
>> ...
>>>
>>> In the problem I am concerned with, I need to compute the Hessian,
>>> but I actually only need the diagonal elements of it. Therefore, it
>>> would be wasted computational effort to calculate all the other
>>> elements. Unfortunately, performance is critical.
>> ...
>>>
>>> The function is of the from R^n -> R .
>>>
>>
...
>>
>> In the mean time, you could try optimizing the tape before computing
>> the Hessian; see
>>     http://www.coin-or.org/CppAD/Doc/optimize.xml
>>
>> If your Hessian is sparse,
>>     http://www.coin-or.org/CppAD/Doc/sparse_hessian.xml
>> may help a lot. Try both boolean and vector of sets for the sparsity
>> pattern; see
>>     http://www.coin-or.org/CppAD/Doc/glossary.xml#Sparsity%20Pattern
>>
>>
>>
>>>
>>>
>>> Yours sincerely,
>>> xxxxxxxxx
>>>
>>>
>