<div dir="ltr">I just looked into Stalingrad - it's cool, but not C or C++. Pytorch also can produce derivatives which are the same type as the original function (Variables, or tensors, depending on the version) and maintains the graph, but, as it's in python, it can be slow.<div><br></div><div>-Andy S.<br><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><br></div></div></div></div><div class="gmail_quote">On Mon, Jun 4, 2018 at 3:20 PM, Andrew Spielberg <span dir="ltr"><<a href="mailto:aespielberg@gmail.com" target="_blank">aespielberg@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Brad - so just to verify, in your example, is there no way to use g to initialize a new ADFun< AD<double> > and then take gradients of that, while maintaining the dependency on the original independent variables?<div><br></div><div>-Andy S.</div><div><div class="h5"><div class="gmail_extra"><br clear="all"><div><div class="m_-1446844117347791670gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><br></div></div></div>
<br><div class="gmail_quote">On Mon, Jun 4, 2018 at 8:05 AM, Brad Bell <span dir="ltr"><<a href="mailto:bradbell@seanet.com" target="_blank">bradbell@seanet.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">When a function f(x) is recorded, CppAD allows for the computation of arbitrary order derivatives of f(x). The multilevel option; e.g., AD< AD<double> > allows one to use arbitrary order derivatives of f(x) to express a function g(x) and then take derivatives of g(x).<br>
<br>
As you have noticed, in this case the function f(x) would be ADFun< AD<double> > and g(x) would be ADFun<double>.<br>
<br>
It is possible that you could do what you are asking for using template meta programmings, but I have never tried.<br>
<br>
I think that the research code stalingrad does automatic differentiation with the derivatives having the same type as the original functions; see<br>
<a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Functional-2DAutoDiff_STALINGRAD&d=DwIGaQ&c=Ngd-ta5yRYsqeUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&r=DI01O1qukvwbwT9tWYo85yppLHUSzhHiWawx-7crnog&m=_25pbHruWRmD1L2XClVRt07j_oFK0I3mg9CGjbh94fw&s=TYrwlh51kdextj7wgvj3pTmut23D6bXYrNHhV8VfGLk&e=" rel="noreferrer" target="_blank">https://urldefense.proofpoint.<wbr>com/v2/url?u=https-3A__github.<wbr>com_Functional-2DAutoDiff_STAL<wbr>INGRAD&d=DwIGaQ&c=Ngd-ta5yRYsq<wbr>eUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc<wbr>4&r=DI01O1qukvwbwT9tWYo85yppLH<wbr>USzhHiWawx-7crnog&m=_25pbHruWR<wbr>mD1L2XClVRt07j_oFK0I3mg9CGjbh9<wbr>4fw&s=TYrwlh51kdextj7wgvj3pTmu<wbr>t23D6bXYrNHhV8VfGLk&e=</a><span><br>
<br>
On 06/03/2018 04:35 PM, Andrew Spielberg wrote:<br>
</span><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span>
Hi all,<br>
<br>
I am trying to come up with a way to get arbitrary levels of AutoDifferentiation using CppAD. Given a function f : R-->R, I would like to be able to differentiate expressions such as, say:<br>
<br>
g(x) = (f'(x) + f''(x)) * f(x)<br>
<br>
in order to get dg/dx.<br>
<br>
Now, obviously, I can try to rewrite this as a single function just in f whose derivative will be equal to dg/dx, but it that is not always possible. To this end, I am aware of the multiple levels example:<br>
<br>
</span><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__coin-2Dor.github.io_CppAD_doc_mul-5Flevel.cpp.htm&d=DwIGaQ&c=Ngd-ta5yRYsqeUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&r=DI01O1qukvwbwT9tWYo85yppLHUSzhHiWawx-7crnog&m=_25pbHruWRmD1L2XClVRt07j_oFK0I3mg9CGjbh94fw&s=7YUvdyn-IO5rWGWTGCWB1L5EUjF6-MgpfQT-8HnYGK8&e=" rel="noreferrer" target="_blank">https://urldefense.proofpoint.<wbr>com/v2/url?u=https-3A__coin-2D<wbr>or.github.io_CppAD_doc_mul-5Fl<wbr>evel.cpp.htm&d=DwIGaQ&c=Ngd-ta<wbr>5yRYsqeUsEDgxhcqsYYY1Xs5ogLxWP<wbr>A_2Wlc4&r=DI01O1qukvwbwT9tWYo8<wbr>5yppLHUSzhHiWawx-7crnog&m=_25p<wbr>bHruWRmD1L2XClVRt07j_oFK0I3mg9<wbr>CGjbh94fw&s=7YUvdyn-IO5rWGWTGC<wbr>WB1L5EUjF6-MgpfQT-8HnYGK8&e=</a> <<a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__coin-2Dor.github.io_CppAD_doc_mul-5Flevel.cpp.htm&d=DwIGaQ&c=Ngd-ta5yRYsqeUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&r=DI01O1qukvwbwT9tWYo85yppLHUSzhHiWawx-7crnog&m=_25pbHruWRmD1L2XClVRt07j_oFK0I3mg9CGjbh94fw&s=7YUvdyn-IO5rWGWTGCWB1L5EUjF6-MgpfQT-8HnYGK8&e=" rel="noreferrer" target="_blank">https://urldefense.proofpoint<wbr>.com/v2/url?u=https-3A__coin-2<wbr>Dor.github.io_CppAD_doc_mul-5F<wbr>level.cpp.htm&d=DwIGaQ&c=Ngd-<wbr>ta5yRYsqeUsEDgxhcqsYYY1Xs5ogLx<wbr>WPA_2Wlc4&r=DI01O1qukvwbwT9tWY<wbr>o85yppLHUSzhHiWawx-7crnog&m=_<wbr>25pbHruWRmD1L2XClVRt07j_oFK0I3<wbr>mg9CGjbh94fw&s=7YUvdyn-IO5rWGW<wbr>TGCWB1L5EUjF6-MgpfQT-8HnYGK8&<wbr>e=</a>><span><br>
<br>
However, as far as I can tell, the number of differentiations here must be known a priori, and the correct number of nestings of ADFun must be chosen up front. In some cases though (say, when dealing with series of unknown length), that number won't be known up front.<br>
<br>
I am wondering if there is a way to get arbitrary levels of autodifferentiation somehow, so that I can treat gradients not unlike an arithmetic function. I was thinking, it might be possible with only two levels of ADFun - every time a ADFun<ADFun<T>> is differentiated, it returns an ADFun<T> - and then maybe this can be rewrapped in a new ADFun< > before being reused. I don't know if this works though.<br>
<br>
Are arbitrary levels possible? If not, is there another way to achieve what I want?<br>
<br>
<br></span>
______________________________<wbr>_________________<br>
CppAD mailing list<br>
<a href="mailto:CppAD@list.coin-or.org" target="_blank">CppAD@list.coin-or.org</a><br>
<a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__list.coin-2Dor.org_mailman_listinfo_cppad&d=DwIGaQ&c=Ngd-ta5yRYsqeUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&r=DI01O1qukvwbwT9tWYo85yppLHUSzhHiWawx-7crnog&m=_25pbHruWRmD1L2XClVRt07j_oFK0I3mg9CGjbh94fw&s=SgKVMryyD8np2uBL5m_GP9Nnjw3k95i_yGWpvJve3k4&e=" rel="noreferrer" target="_blank">https://urldefense.proofpoint.<wbr>com/v2/url?u=https-3A__list.co<wbr>in-2Dor.org_mailman_listinfo_c<wbr>ppad&d=DwIGaQ&c=Ngd-ta5yRYsqeU<wbr>sEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&<wbr>r=DI01O1qukvwbwT9tWYo85yppLHUS<wbr>zhHiWawx-7crnog&m=_25pbHruWRmD<wbr>1L2XClVRt07j_oFK0I3mg9CGjbh94f<wbr>w&s=SgKVMryyD8np2uBL5m_GP9Nnjw<wbr>3k95i_yGWpvJve3k4&e=</a><br>
<br>
</blockquote>
______________________________<wbr>_________________<br>
CppAD mailing list<br>
<a href="mailto:CppAD@list.coin-or.org" target="_blank">CppAD@list.coin-or.org</a><br>
<a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__list.coin-2Dor.org_mailman_listinfo_cppad&d=DwIGaQ&c=Ngd-ta5yRYsqeUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&r=DI01O1qukvwbwT9tWYo85yppLHUSzhHiWawx-7crnog&m=_25pbHruWRmD1L2XClVRt07j_oFK0I3mg9CGjbh94fw&s=SgKVMryyD8np2uBL5m_GP9Nnjw3k95i_yGWpvJve3k4&e=" rel="noreferrer" target="_blank">https://urldefense.proofpoint.<wbr>com/v2/url?u=https-3A__list.co<wbr>in-2Dor.org_mailman_listinfo_c<wbr>ppad&d=DwIGaQ&c=Ngd-ta5yRYsqeU<wbr>sEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&<wbr>r=DI01O1qukvwbwT9tWYo85yppLHUS<wbr>zhHiWawx-7crnog&m=_25pbHruWRmD<wbr>1L2XClVRt07j_oFK0I3mg9CGjbh94f<wbr>w&s=SgKVMryyD8np2uBL5m_GP9Nnjw<wbr>3k95i_yGWpvJve3k4&e=</a><br>
</blockquote></div><br></div></div></div></div>
</blockquote></div><br></div></div></div>