[CppAD] cppad creating a very large tape

[Kasper Kristensen]

Hi,

In case your likelihood function takes the form

l(theta) = sum_i f(x_i,theta)

(where the sum in your case has 6000 terms)
you might be able to reduce tape sizes by only taping a single term and
then manually adding up the derivatives.
This can be done by considering a single term as a function of the
augmented parameter vector (theta,i) and using the CppAD::VecAD type for
the data to ensure that the data subsetting gets taped.
I admit that adding derivatives manually somehow breaks the convenience
of automatic differentiation.

Best regards,
Kasper

  

On Thu, 2009-06-11 at 16:29 +1000, Piers Dunstan wrote:
> Hi,
>     I'm using CPPAD to obtain derivatives of a likelihood function  used 
> to optimise the function.  The likelihood function is complex and has 
> many operations. I have the code working properly and the derivatives 
> are correct. However, when I try to use the code for large data problems 
> the tape size grows very large (ie 20GB) which tends to cause problems 
> on most machines.
> 
> I'm fairly certain that the problem occurs during tape creation since it 
> occurs between calls to  Independent and Dependent. I'm also sure its 
> not a memory leak.
> 
> Much of the data I'm using is constant and used repeatedly (ie  I have a 
> double matrix with dimensions 1100 x 20 that is used 6000 times).  I 
> think the problem comes from the number of operations I have to preform 
> on this matrix. It is multiplied by an AD variable. I think the tape is 
> making multiple copies of the elements in this matrix.
> 
> Is there a way to designate that the matrix is constant and all 
> operations occur on only 1 copy (if this is the problem?)
> 
> Any ideas gratefully accepted.
> 
> Thanks  piers
> 
-- 
Kasper Kristensen <kasper.kristensen at gmail.com>