<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Personally, I would at least want to be able to pass reference to allocated memory. The CppAD definition of a simple vector can be restrictive. For example, most of my mathematical computation is done with the Eigen numerical library, and it would be good if I could just pass a reference to the start of an Eigen vector. Now, I was able to write an Eigen plug-in that makes Eigen vectors qualify as CppAD simple vectors. But that may not always be the case for everyone.<div><br></div><div>Although passing references for output variables would be different from how CppAD works now, it is not all that different from many C libraries (e.g., GSL, Intel CBLAS, etc). Returned values might make more sense if they are returned as expressions, as in Eigen.<br><div><br></div><div>Nevertheless, in general I think it would be useful to minimize memory allocation within CppAD functions, and let the user re-use blocks of memory that are allocated only once.</div><div><br></div><div>I'm looking forward to testing whatever you come up with. Thanks again for all of your work with CppAD.<br><div><br></div><div><br><div><div>On May 9, 2012, at 2:51 PM, <<a href="mailto:bradbell@seanet.com">bradbell@seanet.com</a>></div><div> <<a href="mailto:bradbell@seanet.com">bradbell@seanet.com</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div><blockquote type="cite">If we only want to support one format, this triplet format can work. It<br></blockquote><blockquote type="cite">is intuitive, and there are routines available to convert to and from CSC<br></blockquote><blockquote type="cite">and CSR formats.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">I do have a question about how the Jacobian is returned. If I understand<br></blockquote><blockquote type="cite">the code correctly, CppAD allocates new memory for a new Jacobian/Hessian<br></blockquote><blockquote type="cite">every time the function is called. If I am repeatedly computing gradients<br></blockquote><blockquote type="cite">and Hessians, this is an awful lot of dynamic memory allocation. Wouldn't<br></blockquote><blockquote type="cite">it be more efficient for the user to allocate memory for the output, and<br></blockquote><blockquote type="cite">then pass in a pointer or reference for that array?<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">In that case, the call would be something like:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">f.SparseJacobian(x, p, r, c, *out)<br></blockquote><br>Yes, this would be more efficient, but it would be a marked change from<br>the other cases where the value is returned. Perhaps one should have the<br>option to pass in the allocated memory as you suggest above.<br><br>Perhaps all the CppAD routines should have an interface that uses input<br>and output iterators (for arbitrary containers), instead just simple<br>vectors. Harder for the beginning user, but more versatile, more<br>efficient, and expected by the C++ expert.<br><br><blockquote type="cite"><br></blockquote><blockquote type="cite">Also, will the SparseJacobian / SparseMatrix functions still need to<br></blockquote><blockquote type="cite">allocate a full m-by-n matrix internally? The value of a sparse matrix<br></blockquote><blockquote type="cite">algorithm is not just that the matrix is return in a sparse format, but<br></blockquote><blockquote type="cite">the algorithm itself exploits the sparseness in terms of its memory usage.<br></blockquote><br>The sparse Jacobians and Hessians only allocate space corresponding to one<br>row (forward mode) or one column (reverse mode). Then (using the sparsity)<br>multiple rows or columns are computed as once and then separated.<br><br>On the other hand, the one row or one column is for every intermediate<br>variable in the operation sequence; see size_var<br> <a href="http://www.coin-or.org/CppAD/Doc/seq_property.htm#size_var">http://www.coin-or.org/CppAD/Doc/seq_property.htm#size_var</a><br>not just the independent and dependent variables.<br><br><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> 1. Sparse Hessians and Jacobians (Brad Bell)<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">----------------------------------------------------------------------<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Message: 1<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Date: Tue, 08 May 2012 07:45:05 -0700<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">From: Brad Bell <<a href="mailto:bradbell@seanet.com">bradbell@seanet.com</a>><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">To: <a href="mailto:CppAD@list.coin-or.org">CppAD@list.coin-or.org</a><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Subject: [CppAD] Sparse Hessians and Jacobians<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Message-ID: <<a href="mailto:4FA93171.5040207@seanet.com">4FA93171.5040207@seanet.com</a>><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Content-Type: text/plain; charset=ISO-8859-1; format=flowed<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">I am considering adding sparse Jacobian and Hessians returns to CppAD as<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">per the sparsity discussion under<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> <a href="http://list.coin-or.org/pipermail/cppad/2012q1/thread.html">http://list.coin-or.org/pipermail/cppad/2012q1/thread.html</a><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Reading over the discussion, it seems to me that the format used by<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">ADOL-C is reasonable; i.e. three vectors rind[], cind[], and values[]<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">all of the same length could be used to represent Sparse Hessians and<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Jacobians (where the corresponding element types are size_t, size_t, and<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Base).<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">For example, the sparse Jacobian driver<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> <a href="http://www.coin-or.org/CppAD/Doc/sparse_jacobian.xml">http://www.coin-or.org/CppAD/Doc/sparse_jacobian.xml</a><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">could be extended to have the syntax<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> jac = f.SparseJacobian(x, p, r, c)<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">where<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> x is the point where the Jacobian is being evaluated<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> p is the CppAD sparsity pattern (for the entire Jacobian)<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> r is the row indices<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> c is the column indices<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> jac is the Jacobian values at the corresponding row and column<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">indices.<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Note that the sparsity pattern p can be computed by either<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> <a href="http://www.coin-or.org/CppAD/Doc/forsparsejac.xml">http://www.coin-or.org/CppAD/Doc/forsparsejac.xml</a><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> <a href="http://www.coin-or.org/CppAD/Doc/revsparsejac.xml">http://www.coin-or.org/CppAD/Doc/revsparsejac.xml</a><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Then the users can decide which non-zero indices they want back and in<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">what order. For example, if the Jacobian is know to be symmetric, one<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">only needs to ask for the upper or lower triangle.<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">A similar approach be used for the sparse Hessians.<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> <a href="http://www.coin-or.org/CppAD/Doc/sparse_hessian.xml">http://www.coin-or.org/CppAD/Doc/sparse_hessian.xml</a><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">------------------------------<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">_______________________________________________<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">CppAD mailing list<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><a href="mailto:CppAD@list.coin-or.org">CppAD@list.coin-or.org</a><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><a href="http://list.coin-or.org/mailman/listinfo/cppad">http://list.coin-or.org/mailman/listinfo/cppad</a><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">End of CppAD Digest, Vol 56, Issue 1<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">************************************<br></blockquote></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Michael Braun<br></blockquote><blockquote type="cite">MIT Sloan School of Management<br></blockquote><blockquote type="cite"><a href="mailto:braunm@mit.edu">braunm@mit.edu</a><br></blockquote><blockquote type="cite">--------------------------------------------------<br></blockquote><blockquote type="cite">View my research at<br></blockquote><blockquote type="cite"><a href="http://braunm.scripts.mit.edu/">http://braunm.scripts.mit.edu/</a><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">_______________________________________________<br></blockquote><blockquote type="cite">CppAD mailing list<br></blockquote><blockquote type="cite"><a href="mailto:CppAD@list.coin-or.org">CppAD@list.coin-or.org</a><br></blockquote><blockquote type="cite"><a href="http://list.coin-or.org/mailman/listinfo/cppad">http://list.coin-or.org/mailman/listinfo/cppad</a><br></blockquote><blockquote type="cite"><br></blockquote><br><br></div></blockquote></div><br><div>
<span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div>-------------------------------------------<br>Michael Braun<br>Associate Professor of Management Science (Marketing Group)<br>MIT Sloan School of Management<br>100 Main St.., E62-535<br>Cambridge, MA 02139<br><a href="mailto:braunm@mit.edu">braunm@mit.edu</a><br>617-253-3436</div><div><br></div></div></span><br class="Apple-interchange-newline"></span><br class="Apple-interchange-newline">
</div>
<br></div></div></div></body></html>