[CppAD] Arbitrary levels of autodifferentiation

[Andrew Spielberg]

I just looked into Stalingrad - it's cool, but not C or C++.  Pytorch also
can produce derivatives which are the same type as the original function
(Variables, or tensors, depending on the version) and maintains the graph,
but, as it's in python, it can be slow.

-Andy S.


On Mon, Jun 4, 2018 at 3:20 PM, Andrew Spielberg <aespielberg at gmail.com>
wrote:

> Hi Brad - so just to verify, in your example, is there no way to use g to
> initialize a new ADFun< AD<double> > and then take gradients of that, while
> maintaining the dependency on the original independent variables?
>
> -Andy S.
>
>
>
> On Mon, Jun 4, 2018 at 8:05 AM, Brad Bell <bradbell at seanet.com> wrote:
>
>> When a function f(x) is recorded, CppAD allows for the computation of
>> arbitrary order derivatives of f(x). The multilevel option; e.g., AD<
>> AD<double> > allows one to use arbitrary order derivatives of f(x) to
>> express a function g(x) and then take derivatives of g(x).
>>
>> As you have noticed, in this case the function f(x) would be ADFun<
>> AD<double> > and g(x) would be ADFun<double>.
>>
>> It is possible that you could do what you are asking for using template
>> meta programmings, but I have never tried.
>>
>> I think that the research code stalingrad does automatic differentiation
>> with the derivatives having the same type as the original functions; see
>>         https://urldefense.proofpoint.com/v2/url?u=https-3A__github.
>> com_Functional-2DAutoDiff_STALINGRAD&d=DwIGaQ&c=Ngd-ta5yRYsq
>> eUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&r=DI01O1qukvwbwT9tWYo85yppLH
>> USzhHiWawx-7crnog&m=_25pbHruWRmD1L2XClVRt07j_oFK0I3mg9CGjbh9
>> 4fw&s=TYrwlh51kdextj7wgvj3pTmut23D6bXYrNHhV8VfGLk&e=
>>
>> On 06/03/2018 04:35 PM, Andrew Spielberg wrote:
>>
>>> Hi all,
>>>
>>> I am trying to come up with a way to get arbitrary levels of
>>> AutoDifferentiation using CppAD.  Given a function f : R-->R, I would like
>>> to be able to differentiate expressions such as, say:
>>>
>>> g(x) = (f'(x) + f''(x)) * f(x)
>>>
>>> in order to get dg/dx.
>>>
>>> Now, obviously, I can try to rewrite this as a single function just in f
>>> whose derivative will be equal to dg/dx, but it that is not always
>>> possible.  To this end, I am aware of the multiple levels example:
>>>
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__coin-2D
>>> or.github.io_CppAD_doc_mul-5Flevel.cpp.htm&d=DwIGaQ&c=Ngd-ta
>>> 5yRYsqeUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&r=DI01O1qukvwbwT9tWYo8
>>> 5yppLHUSzhHiWawx-7crnog&m=_25pbHruWRmD1L2XClVRt07j_oFK0I3mg9
>>> CGjbh94fw&s=7YUvdyn-IO5rWGWTGCWB1L5EUjF6-MgpfQT-8HnYGK8&e= <
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__coin-2
>>> Dor.github.io_CppAD_doc_mul-5Flevel.cpp.htm&d=DwIGaQ&c=Ngd-
>>> ta5yRYsqeUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&r=DI01O1qukvwbwT9tWY
>>> o85yppLHUSzhHiWawx-7crnog&m=_25pbHruWRmD1L2XClVRt07j_oFK0I3
>>> mg9CGjbh94fw&s=7YUvdyn-IO5rWGWTGCWB1L5EUjF6-MgpfQT-8HnYGK8&e=>
>>>
>>> However, as far as I can tell, the number of differentiations here must
>>> be known a priori, and the correct number of nestings of ADFun must be
>>> chosen up front.  In some cases though (say, when dealing with series of
>>> unknown length), that number won't be known up front.
>>>
>>> I am wondering if there is a way to get arbitrary levels of
>>> autodifferentiation somehow, so that I can treat gradients not unlike an
>>> arithmetic function.  I was thinking, it might be possible with only two
>>> levels of ADFun - every time a ADFun<ADFun<T>> is differentiated, it
>>> returns an ADFun<T> - and then maybe this can be rewrapped in a new ADFun<
>>> > before being reused.  I don't know if this works though.
>>>
>>> Are arbitrary levels possible?  If not, is there another way to achieve
>>> what I want?
>>>
>>>
>>> _______________________________________________
>>> CppAD mailing list
>>> CppAD at list.coin-or.org
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__list.co
>>> in-2Dor.org_mailman_listinfo_cppad&d=DwIGaQ&c=Ngd-ta5yRYsqeU
>>> sEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&r=DI01O1qukvwbwT9tWYo85yppLHUS
>>> zhHiWawx-7crnog&m=_25pbHruWRmD1L2XClVRt07j_oFK0I3mg9CGjbh94f
>>> w&s=SgKVMryyD8np2uBL5m_GP9Nnjw3k95i_yGWpvJve3k4&e=
>>>
>>> _______________________________________________
>> CppAD mailing list
>> CppAD at list.coin-or.org
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__list.co
>> in-2Dor.org_mailman_listinfo_cppad&d=DwIGaQ&c=Ngd-ta5yRYsqeU
>> sEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&r=DI01O1qukvwbwT9tWYo85yppLHUS
>> zhHiWawx-7crnog&m=_25pbHruWRmD1L2XClVRt07j_oFK0I3mg9CGjbh94f
>> w&s=SgKVMryyD8np2uBL5m_GP9Nnjw3k95i_yGWpvJve3k4&e=
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://list.coin-or.org/pipermail/cppad/attachments/20180604/973f6be6/attachment-0001.html>