<html>
  <head>
    <meta content="text/html; charset=ISO-8859-1"
      http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <div class="moz-cite-prefix">Sorry.&nbsp; By the "kernel", I mean the
      kernel of the Jacobian of f(x):<br>
      <br>
      All u in R^n such that&nbsp;&nbsp; df/dx * u = 0<br>
      <br>
      These u form the tangent space of the constraint "surface".<br>
      You could, for example, use SVD to find an orthonormal basis of
      the kernel.<br>
      <br>
      Cheers,<br>
      Norm<br>
      <br>
      <br>
      On 04/09/2013 01:11 AM, Ingrid Hallen wrote:<br>
    </div>
    <blockquote cite="mid:DUB107-W799D092ABF25D599F14CED2C60@phx.gbl"
      type="cite">
      <style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 12pt;
font-family:Calibri
}
--></style>
      <div dir="ltr">Ok, I think I kind of understand the idea. However,
        doesn't it require<br>
        that one solves equations of the form f(x) = 0,&nbsp; where f define<br>
        some of the constraints, for each iterate x to identify the
        kernel? <br>
        It seems related to the idea of eliminating equality constraints
        (as<br>
        in e.g. p. 132 in Convex Optimization by Boyd and Vandenberge),<br>
        which, as I did not mention :), I don't want to do.<br>
        <br>
        <br>
        Kind regards,<br>
        <br>
        Ingrid<br>
        <br>
        <br>
        <br>
        <br>
        <br>
        <br>
        <div>
          <hr id="stopSpelling">Date: Mon, 8 Apr 2013 13:14:39 -0700<br>
          From: <a class="moz-txt-link-abbreviated" href="mailto:normvcr@telus.net">normvcr@telus.net</a><br>
          To: <a class="moz-txt-link-abbreviated" href="mailto:ingridhallen@hotmail.com">ingridhallen@hotmail.com</a><br>
          CC: <a class="moz-txt-link-abbreviated" href="mailto:adol-c@list.coin-or.org">adol-c@list.coin-or.org</a><br>
          Subject: Re: [ADOL-C] Evaluating Hessian of Lagrangian<br>
          <br>
          <div class="ecxmoz-cite-prefix">For the second point,
            specially since there are almost as many<br>
            constraints as variables, it may be worthwhile to take the
            Hessian<br>
            with respect to only the directions in the kernel of the
            constraints.<br>
            For example, if you are working in 20-dimensional Euclidean
            space,<br>
            with 18 constraints, that leaves an essentially
            2-dimensional optimization<br>
            problem, and you only really need a 2x2 Hessian, not a 20x20
            Hessian.<br>
            You can choose two orthogonal directions in the kernel of
            the constraints,<br>
            and construct the 2x2 Hessian of Lagrangian wrt them,
            viewing this either<br>
            as directional derivatives or change of coordinates.<br>
            <br>
            I don't know what amount of SW changes this would need, but
            it<br>
            seems it would be generally useful.<br>
            <br>
            Cheers,<br>
            Norm<br>
            <br>
            <br>
            On 04/08/2013 06:38 AM, Ingrid Hallen wrote:<br>
          </div>
          <blockquote
            cite="mid:DUB107-W2630FC5BF85DADA09BC5A6D2C50@phx.gbl">
            <style><!--
.ExternalClass .ecxhmmessage P {
padding:0px;
}

.ExternalClass body.ecxhmmessage {
font-size:12pt;
font-family:Calibri;
}

--></style>
            <div dir="ltr">Thanks for the suggestions!<br>
              <br>
              Regarding the first, I think it might not be so efficient
              for my problem, having<br>
              almost as many constraints as variables. But I might give
              it a try. Perhaps<br>
              one can modify the code for sparse_hess in some clever way
              ...<br>
              <br>
              Regarding the second I am unfortunately clueless as to how
              one could<br>
              exploit that fact.<br>
              <br>
              Kind regards,<br>
              <br>
              Ingrid<br>
              <br>
              <br>
              <div>
                <hr id="ecxstopSpelling">Date: Sun, 7 Apr 2013 15:31:50
                -0700<br>
                From: <a moz-do-not-send="true"
                  class="ecxmoz-txt-link-abbreviated"
                  href="mailto:normvcr@telus.net">normvcr@telus.net</a><br>
                To: <a moz-do-not-send="true"
                  class="ecxmoz-txt-link-abbreviated"
                  href="mailto:adol-c@list.coin-or.org">adol-c@list.coin-or.org</a><br>
                Subject: Re: [ADOL-C] Evaluating Hessian of Lagrangian<br>
                <br>
                <div class="ecxmoz-cite-prefix">Perhaps you can trace L
                  as a function of both x and lambda.<br>
                  Then when you need the hessian,<br>
                  calculate the hessian only with respect to x.<br>
                  Not sure if this is better/worse than what you
                  suggested.<br>
                  <br>
                  When using the Hessian of the Lagrangian, you will
                  eventually be restricting<br>
                  attention to the kernel of your constraints.&nbsp; Do you
                  know if this fact<br>
                  would offer a simplification to how the hessian could
                  be computed?<br>
                  <br>
                  Norm<br>
                  <br>
                  <br>
                  On 04/05/2013 04:02 AM, Ingrid Hallen wrote:<br>
                </div>
                <blockquote
                  cite="mid:DUB107-W59AC8B59AD0A9DEA3D6DB2D2DA0@phx.gbl">
                  <style><!--
.ExternalClass .ecxhmmessage P {
padding:0px;
}

.ExternalClass body.ecxhmmessage {
font-size:12pt;
font-family:Calibri;
}


--></style>
                  <div dir="ltr">Hi,<br>
                    <br>
                    I'm doing non-linear optimization with IPOPT. For
                    this, I'm using ADOL-C<br>
                    to compute the Hessian of the Lagrangian<br>
                    <br>
                    L(x,lambda) = f(x) + sum_{i}lambda_{i}h_{i}(x),<br>
                    <br>
                    where x are the variables, lambda the Lagrange
                    multipliers and <br>
                    f(x) and h_{i}(x) objective and constraint
                    functions.<br>
                    <br>
                    What I'm doing in my code is the following (omitting
                    details):<br>
                    <br>
                    // **********************<br>
                    <br>
                    // Trace Lagrangian function<br>
                    trace_on(tag);<br>
                    <br>
                    for(i=0;i&lt;n;i++) {<br>
                    &nbsp;&nbsp;&nbsp; xad[i] &lt;&lt;= x[i];<br>
                    }<br>
                    <br>
                    Lagrangian(xad, lambda);<br>
                    <br>
                    Lad &gt;&gt;=L;<br>
                    <br>
                    trace_off();<br>
                    <br>
                    // Evaluate Hessian of the Lagrangian<br>
                    repeat = 0;<br>
sparse_hess(tag,n,repeat,x,&amp;nnz,&amp;rind,&amp;cind,&amp;values,&amp;options)<br>
                    <br>
                    // ***********************<br>
                    <br>
                    This works fine, but is not so efficient. One reason
                    is that, since lambda changes, <br>
                    the Lagrangian function has to be retaped every time
                    the Hessian is needed and so it <br>
                    appears that I cannot set repeat = 1 when calling
                    sparse_hess.<br>
                    <br>
                    One way to circumvent this problem could perhaps be
                    to trace the objective<br>
                    and constraint functions individually and then
                    construct the Hessian of<br>
                    the Lagrangian using multiple calls to sparse_hess,
                    but is there a<br>
                    more convenient way to do it? <br>
                    <br>
                    Sincerely,<br>
                    <br>
                    Ingrid<br>
                    <br>
                  </div>
                  <br>
                  <fieldset class="ecxmimeAttachmentHeader"></fieldset>
                  <br>
                  <pre>_______________________________________________
ADOL-C mailing list
<a moz-do-not-send="true" class="ecxmoz-txt-link-abbreviated" href="mailto:ADOL-C@list.coin-or.org">ADOL-C@list.coin-or.org</a>
<a moz-do-not-send="true" class="ecxmoz-txt-link-freetext" href="http://list.coin-or.org/mailman/listinfo/adol-c" target="_blank">http://list.coin-or.org/mailman/listinfo/adol-c</a></pre>
                </blockquote>
                <br>
                <br>
                _______________________________________________ ADOL-C
                mailing list <a moz-do-not-send="true"
                  class="ecxmoz-txt-link-abbreviated"
                  href="mailto:ADOL-C@list.coin-or.org">ADOL-C@list.coin-or.org</a>
                <a moz-do-not-send="true"
                  class="ecxmoz-txt-link-freetext"
                  href="http://list.coin-or.org/mailman/listinfo/adol-c"
                  target="_blank">http://list.coin-or.org/mailman/listinfo/adol-c</a></div>
            </div>
          </blockquote>
          <br>
        </div>
      </div>
    </blockquote>
    <br>
  </body>
</html>