<HTML><HEAD></HEAD>
<BODY dir=ltr>
<DIV dir=ltr>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: 'Calibri'; COLOR: #000000">
<DIV>I’m not used to seeing Ipopt’s timing statistics report larger wall time
than CPU time, usually it’s the other way around when I’m using one of the
openmp linear solvers. I have heard that Python forking can interact very poorly
with multithreaded BLAS, so something odd might be happening.</DIV>
<DIV> </DIV>
<DIV>Definitely profile your function evaluations. Depending on how complicated
your nonlinearities are and how much code we’re talking about, you may want to
pursue implementing your function evaluations in something faster than Python,
particularly something that can give you efficient automatic differentiation. If
you’re using AMPL, GAMS, JuMP, Pyomo (which outputs models to .nl files like
AMPL), or OSiL, then generally function evaluations should only be taking around
5-20% of the wall time in Ipopt.</DIV>
<DIV> </DIV>
<DIV>-Tony</DIV>
<DIV> </DIV>
<DIV
style='FONT-SIZE: small; TEXT-DECORATION: none; FONT-FAMILY: "Calibri"; FONT-WEIGHT: normal; COLOR: #000000; FONT-STYLE: normal; DISPLAY: inline'>
<DIV style="FONT: 10pt tahoma">
<DIV><FONT size=3 face=Calibri></FONT> </DIV>
<DIV style="BACKGROUND: #f5f5f5">
<DIV style="font-color: black"><B>From:</B> <A title=ipoptuser@gmail.com
href="mailto:ipoptuser@gmail.com">Ipopt User</A> </DIV>
<DIV><B>Sent:</B> Thursday, November 13, 2014 10:25 AM</DIV>
<DIV><B>To:</B> <A title=ipopt@list.coin-or.org
href="mailto:ipopt@list.coin-or.org">ipopt@list.coin-or.org</A> </DIV>
<DIV><B>Subject:</B> Re: [Ipopt] Low # iters, ensuring that solution remains
feasible</DIV></DIV></DIV>
<DIV> </DIV></DIV>
<DIV
style='FONT-SIZE: small; TEXT-DECORATION: none; FONT-FAMILY: "Calibri"; FONT-WEIGHT: normal; COLOR: #000000; FONT-STYLE: normal; DISPLAY: inline'>
<DIV dir=ltr><SPAN style="FONT-SIZE: 13px; FONT-FAMILY: arial,sans-serif">To see
where those 15 seconds are spent, look at the wall time in the advanced timing
statistics: 14.946 out of 15.734 seconds are spent in function evaluations. This
seems normal for the quasi newton method if you have only a few
constraints.</SPAN>
<DIV style="FONT-SIZE: 13px; FONT-FAMILY: arial,sans-serif"><FONT size=3
face=Calibri></FONT> </DIV>
<DIV style="FONT-SIZE: 13px; FONT-FAMILY: arial,sans-serif">Why does the
optimization terminate early?</DIV></DIV>
<DIV class=gmail_extra>
<DIV> </DIV>
<DIV class=gmail_quote>On Thu, Nov 13, 2014 at 1:24 PM, Ipopt User <SPAN
dir=ltr><<A href="mailto:ipoptuser@gmail.com"
target=_blank>ipoptuser@gmail.com</A>></SPAN> wrote:<BR>
<BLOCKQUOTE class=gmail_quote
style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">
<DIV dir=ltr>To see where those 15 seconds are spent, look at the wall time in
the advanced timing statistics: 14.946 out of 15.734 seconds are spent in
function evaluations. This seems normal for the quasi newton method if you
have only a few constraints.
<DIV> </DIV>
<DIV>Why does the optimization terminate early?</DIV></DIV>
<DIV class=HOEnZb>
<DIV class=h5>
<DIV class=gmail_extra>
<DIV> </DIV>
<DIV class=gmail_quote>On Thu, Nov 13, 2014 at 12:45 PM, John Schulman <SPAN
dir=ltr><<A href="mailto:john.d.schulman@gmail.com"
target=_blank>john.d.schulman@gmail.com</A>></SPAN> wrote:<BR>
<BLOCKQUOTE class=gmail_quote
style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">
<DIV dir=ltr>
<DIV>I haven't had much luck yet setting the initial Lagrange
multipliers.</DIV>
<DIV>One hack that helps a bit is to redefine the objective function so that
it has value +inf when the constraint violation is greater than some
threshold.</DIV>
<DIV>Of course, this severely impedes the progress of the optimizer and
causes it to emit a lot of warnings.</DIV>
<DIV>But maybe rescaling or reshaping the objective function is the way to
go here.</DIV>
<DIV> </DIV>
<DIV>Yup, the problem 50000 dimensional, and there's no banded-diagonal
structure or anything. I think the main bottleneck should be function and
gradient evaluations, not the linear algebra. I looked at the timing stats
and something is a bit off. The optimization takes about 15 seconds but the
stats only account for about 1 second. Maybe because I'm using fork-based
parallelism (via python multiprocessing) to compute the gradients? Anyway,
if it's indeed true that linear algebra is taking .5 seconds or so, then
that's insignificant.</DIV>
<DIV> </DIV>
<DIV> </DIV>
<DIV> </DIV>
<DIV>
<DIV>Number of objective function
evaluations
= 70</DIV>
<DIV>Number of objective gradient
evaluations
= 51</DIV>
<DIV>Number of equality constraint
evaluations
= 0</DIV>
<DIV>Number of inequality constraint
evaluations = 70</DIV>
<DIV>Number of equality constraint Jacobian evaluations =
0</DIV>
<DIV>Number of inequality constraint Jacobian evaluations = 51</DIV>
<DIV>Number of Lagrangian Hessian
evaluations
= 0</DIV>
<DIV>Total CPU secs in IPOPT (w/o function evaluations)
= 0.591</DIV>
<DIV>Total CPU secs in NLP function
evaluations
= 0.439</DIV>
<DIV> </DIV>
<DIV> </DIV>
<DIV>Timing Statistics:</DIV>
<DIV> </DIV>
<DIV>OverallAlgorithm....................:
1.030 (sys: 0.064
wall: 15.734)</DIV>
<DIV>PrintProblemStatistics.............:
0.000 (sys: 0.000
wall: 0.000)</DIV>
<DIV>InitializeIterates.................:
0.008 (sys: 0.001
wall: 0.222)</DIV>
<DIV>UpdateHessian......................:
0.051 (sys: 0.002
wall: 0.055)</DIV>
<DIV>OutputIteration....................:
0.001 (sys: 0.001
wall: 0.002)</DIV>
<DIV>UpdateBarrierParameter.............:
0.353 (sys: 0.022
wall: 0.378)</DIV>
<DIV>ComputeSearchDirection.............:
0.133 (sys: 0.005
wall: 0.137)</DIV>
<DIV>ComputeAcceptableTrialPoint........:
0.176 (sys: 0.017
wall: 7.201)</DIV>
<DIV>AcceptTrialPoint...................:
0.004 (sys: 0.000
wall: 0.004)</DIV>
<DIV>CheckConvergence...................:
0.304 (sys: 0.015
wall: 7.735)</DIV>
<DIV>PDSystemSolverTotal.................:
0.436 (sys: 0.020
wall: 0.458)</DIV>
<DIV>PDSystemSolverSolveOnce............:
0.403 (sys: 0.019
wall: 0.422)</DIV>
<DIV>ComputeResiduals...................:
0.026 (sys: 0.001
wall: 0.027)</DIV>
<DIV>StdAugSystemSolverMultiSolve.......:
0.187 (sys: 0.009
wall: 0.197)</DIV>
<DIV>LinearSystemScaling................:
0.000 (sys: 0.000
wall: 0.000)</DIV>
<DIV>LinearSystemSymbolicFactorization..:
0.000 (sys: 0.000
wall: 0.001)</DIV>
<DIV>LinearSystemFactorization..........:
0.030 (sys: 0.000
wall: 0.031)</DIV>
<DIV>LinearSystemBackSolve..............:
0.133 (sys: 0.000
wall: 0.133)</DIV>
<DIV>LinearSystemStructureConverter.....:
0.000 (sys: 0.000
wall: 0.000)</DIV>
<DIV>
LinearSystemStructureConverterInit: 0.000
(sys: 0.000
wall: 0.000)</DIV>
<DIV>QualityFunctionSearch...............:
0.049 (sys: 0.005
wall: 0.054)</DIV>
<DIV>TryCorrector........................:
0.000 (sys: 0.000
wall: 0.000)</DIV>
<DIV>Task1...............................:
0.014 (sys: 0.002
wall: 0.017)</DIV>
<DIV>Task2...............................:
0.028 (sys: 0.000
wall: 0.028)</DIV>
<DIV>Task3...............................:
0.002 (sys: 0.000
wall: 0.003)</DIV>
<DIV>Task4...............................:
0.000 (sys: 0.000
wall: 0.000)</DIV>
<DIV>Task5...............................:
0.002 (sys: 0.000
wall: 0.003)</DIV>
<DIV>Function Evaluations................:
0.439 (sys: 0.031
wall: 14.946)</DIV>
<DIV>Objective function.................:
0.069 (sys: 0.007
wall: 3.589)</DIV>
<DIV>Objective function gradient........:
0.149 (sys: 0.008
wall: 3.946)</DIV>
<DIV>Equality constraints...............:
0.000 (sys: 0.000
wall: 0.000)</DIV>
<DIV>Inequality constraints.............:
0.077 (sys: 0.009
wall: 3.686)</DIV>
<DIV>Equality constraint Jacobian.......:
0.000 (sys: 0.000
wall: 0.000)</DIV>
<DIV>Inequality constraint Jacobian.....:
0.145 (sys: 0.007
wall: 3.725)</DIV>
<DIV>Lagrangian Hessian.................:
0.000 (sys: 0.000
wall: 0.000)</DIV></DIV>
<DIV> </DIV></DIV></BLOCKQUOTE></DIV></DIV></DIV></DIV></BLOCKQUOTE></DIV></DIV></DIV></DIV></DIV></BODY></HTML>