<HTML><HEAD></HEAD>
<BODY dir=ltr>
<DIV dir=ltr>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: 'Calibri'; COLOR: #000000">
<DIV>Yikes, you should’ve said you have a dense Hessian, that’s bad news. You
really have 50000 x 50000 all-to-all nonlinear interaction terms? I hope you’re
using OpenBLAS or MKL. Depending on the results of print_timing_statistics, also
a good idea to experiment with as many different linear solvers as you can
get.</DIV>
<DIV> </DIV>
<DIV
style='FONT-SIZE: small; TEXT-DECORATION: none; FONT-FAMILY: "Calibri"; FONT-WEIGHT: normal; COLOR: #000000; FONT-STYLE: normal; DISPLAY: inline'>
<DIV style="FONT: 10pt tahoma">
<DIV> </DIV>
<DIV style="BACKGROUND: #f5f5f5">
<DIV style="font-color: black"><B>From:</B> <A title=john.d.schulman@gmail.com
href="mailto:john.d.schulman@gmail.com">John Schulman</A> </DIV>
<DIV><B>Sent:</B> Wednesday, November 12, 2014 10:47 PM</DIV>
<DIV><B>To:</B> <A title=kelman@berkeley.edu
href="mailto:kelman@berkeley.edu">Tony Kelman</A> </DIV>
<DIV><B>Cc:</B> <A title=ipopt@list.coin-or.org
href="mailto:ipopt@list.coin-or.org">ipopt@list.coin-or.org</A> </DIV>
<DIV><B>Subject:</B> Re: [Ipopt] Low # iters, ensuring that solution remains
feasible</DIV></DIV></DIV>
<DIV> </DIV></DIV>
<DIV
style='FONT-SIZE: small; TEXT-DECORATION: none; FONT-FAMILY: "Calibri"; FONT-WEIGHT: normal; COLOR: #000000; FONT-STYLE: normal; DISPLAY: inline'>
<DIV dir=ltr>Hi Tony,
<DIV> </DIV>
<DIV>Funny to get a reply from someone so close by after launching a message out
to the colossal internet.</DIV>
<DIV> </DIV>
<DIV>I wasn't warm-starting at all--that's a great suggestion. Hopefully the
Lagrange multipliers won't change too much between iterations. Otherwise, there
may be some problem-specific ways to guess a reasonable Lagrange multiplier
here.</DIV>
<DIV> </DIV>
<DIV>Regarding speed & algorithm choice, BFGS seems like the right fit here,
and in this problem it's not really practical to compute the hessian (which is
dense).</DIV>
<DIV>But I'll be sure to check out print_timing_statistics -- I'm new to Ipopt
and haven't found these handy tools yet.</DIV>
<DIV> </DIV>
<DIV>John</DIV>
<DIV> </DIV>
<DIV class=gmail_extra>
<DIV> </DIV>
<DIV class=gmail_quote>On Wed, Nov 12, 2014 at 10:19 PM, Tony Kelman <SPAN
dir=ltr><<A href="mailto:kelman@berkeley.edu"
target=_blank>kelman@berkeley.edu</A>></SPAN> wrote:<BR>
<BLOCKQUOTE class=gmail_quote
style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">
<DIV dir=ltr>
<DIV dir=ltr>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: 'Calibri'; COLOR: #000000">
<DIV>Hi John, good to see groups next door also using Ipopt.</DIV>
<DIV> </DIV>
<DIV>Generally speaking this is a pretty hard thing to do with an
interior-point method. Are you warm-starting the dual variables as well, or
just the primal? That may help, but it depends how closely related the
subproblems are. I’d also avoid doing quasi-newton hessian approximations if
you have a speed-critical application, you’ll get better convergence in most
cases if you are using a modeling tool that can provide exact Hessians. Have
you looked at the breakdown of computation time from
print_timing_statistics?</DIV>
<DIV> </DIV>
<DIV>-Tony</DIV>
<DIV> </DIV>
<DIV
style='FONT-SIZE: small; TEXT-DECORATION: none; FONT-FAMILY: "Calibri"; FONT-WEIGHT: normal; COLOR: #000000; FONT-STYLE: normal; DISPLAY: inline'>
<DIV style="FONT: 10pt tahoma">
<DIV><FONT size=3 face=Calibri></FONT> </DIV>
<DIV style="BACKGROUND: #f5f5f5">
<DIV><B>From:</B> <A title=john.d.schulman@gmail.com
href="mailto:john.d.schulman@gmail.com" target=_blank>John Schulman</A> </DIV>
<DIV><B>Sent:</B> Wednesday, November 12, 2014 10:14 PM</DIV>
<DIV><B>To:</B> <A title=ipopt@list.coin-or.org
href="mailto:ipopt@list.coin-or.org" target=_blank>ipopt@list.coin-or.org</A>
</DIV>
<DIV><B>Subject:</B> Re: [Ipopt] Low # iters, ensuring that solution remains
feasible</DIV></DIV></DIV>
<DIV> </DIV></DIV>
<DIV
style='FONT-SIZE: small; TEXT-DECORATION: none; FONT-FAMILY: "Calibri"; FONT-WEIGHT: normal; COLOR: #000000; FONT-STYLE: normal; DISPLAY: inline'>
<DIV>
<DIV>
<DIV dir=ltr>Oops, "wildly feasible" in the first paragraph should be "wildly
infeasible"</DIV>
<DIV class=gmail_extra>
<DIV> </DIV>
<DIV class=gmail_quote>On Wed, Nov 12, 2014 at 10:06 PM, John Schulman <SPAN
dir=ltr><<A href="mailto:john.d.schulman@gmail.com"
target=_blank>john.d.schulman@gmail.com</A>></SPAN> wrote:<BR>
<BLOCKQUOTE class=gmail_quote
style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">
<DIV dir=ltr>
<DIV>Short: </DIV>
<DIV> </DIV>
<DIV>I am calling Ipopt repeatedly to solve a series of subproblems.</DIV>
<DIV>For each subproblem, Ipopt is initialized with a feasible solution, and
max_iter is set to 50 or so. </DIV>
<DIV>The optimization terminates early, and often this intermediate solution
is wildly feasible.</DIV>
<DIV>I'm wondering if there are any settings that will ensure that the
result is nearly feasible.</DIV>
<DIV> </DIV>
<DIV>Longer:</DIV>
<DIV> </DIV>I am using Ipopt to solve a series of subproblems of the
form
<DIV>minimize f(x), subject to g(x) < delta,</DIV>
<DIV>Here g is a distance function of sorts, measuring Distance(x_0,x),
where x_0 is the initialization.</DIV>
<DIV>So the the initial point x_0 is feasible.</DIV>
<DIV>x has dimension 50000 or so, so I am using hessian_approximation with
limited memory.</DIV>
<DIV>
<DIV> </DIV>
<DIV>I need to keep to a low number of iterations, say 50 or 100, so the
overall computation time remains reasonable.</DIV></DIV>
<DIV>It's not essential at all that the solution generated is optimal; I
just want to improve the objective as much as possible while remaining
feasible.</DIV>
<DIV> </DIV>
<DIV>I tried fiddling with the barrier parameters but didn't have any
luck.</DIV>
<DIV>Any suggestions?<BR>Thanks in advance for your time.</DIV><SPAN><FONT
color=#888888>
<DIV> </DIV>
<DIV>John</DIV>
<DIV><FONT color=#000000></FONT> </DIV>
<DIV> </DIV>
<DIV> </DIV>
<DIV> </DIV></FONT></SPAN></DIV></BLOCKQUOTE></DIV>
<DIV> </DIV></DIV></DIV></DIV>
<HR>
_______________________________________________<BR>Ipopt mailing list<BR><A
href="mailto:Ipopt@list.coin-or.org"
target=_blank>Ipopt@list.coin-or.org</A><BR><A
href="http://list.coin-or.org/mailman/listinfo/ipopt"
target=_blank>http://list.coin-or.org/mailman/listinfo/ipopt</A><BR></DIV></DIV></DIV></DIV></BLOCKQUOTE></DIV>
<DIV> </DIV></DIV></DIV></DIV></DIV></DIV></BODY></HTML>