[Ipopt] How to warm start dual variables in IPOPT?
Naiyuan Chiang
sorakid507 at gmail.com
Sat May 7 09:57:32 EDT 2016
I think the key points are:
A) switch off scaling function:
nlp_scaling_method none
(The default way evaluates the scaling factor based on the initial
gradient.)
B) Telling Ipopt to do warm start before the second solve:
warm_start_init_point = yes
warm_start_bound_frac = 1e-16
warm_start_bound_push = 1e-16
warm_start_mult_bound_push = 1e-16
warm_start_slack_bound_frac = 1e-16
warm_start_slack_bound_push = 1e-16
C) Use exact-hessian to solve the problem. If L-bfgs is applied, different
hessian approximation would be used in the 2nd run. This is due to that
L-bfgs uses historic information in the 1st solve.
I attached two results below and you can see that the second run has exact
same result as the solution at the first run.
Cheers,
Nai-Yuan
Senior Scientist
United Technologies Research Center
CT, USA
List of user-set options:
Name Value used
bound_frac = 0.0001 yes
bound_push = 0.0001 yes
limited_memory_max_history = 6 no
linear_solver = ma27 yes
max_iter = 100 yes
mu_init = 0.1 yes
mu_strategy = monotone yes
nlp_scaling_method = none yes
output_file = ipopt.out yes
print_level = 5 yes
print_user_options = yes yes
structured_bfgs_linearalg = yes no
tol = 0.0001 yes
use_cholfact_lowranksol = no no
******************************************************************************
This program contains Ipopt, a library for large-scale nonlinear
optimization.
Ipopt is released as open source code under the Eclipse Public License
(EPL).
For more information visit http://projects.coin-or.org/Ipopt
******************************************************************************
This is Ipopt version 3.12.0, running with linear solver ma27.
Number of nonzeros in equality constraint Jacobian...: 164
Number of nonzeros in inequality constraint Jacobian.: 139
Number of nonzeros in Lagrangian Hessian.............: 252
Total number of variables............................: 23
variables with only lower bounds: 0
variables with lower and upper bounds: 23
variables with only upper bounds: 0
Total number of equality constraints.................: 18
Total number of inequality constraints...............: 21
inequality constraints with only lower bounds: 0
inequality constraints with lower and upper bounds: 21
inequality constraints with only upper bounds: 0
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du
alpha_pr ls InReg SiReg
0 8.3631250e+03 1.55e+00 3.53e+03 -1.0 0.00e+00 - 0.00e+00
0.00e+00 0 0 0
1 7.3211706e+03 1.08e+00 4.72e+06 -1.0 4.59e-01 - 7.74e-01
2.98e-01f 1 0 0
2 7.3108635e+03 1.07e+00 4.67e+06 -1.0 1.23e+01 4.0 4.16e-04
7.24e-03f 1 5 0
3 7.2686586e+03 1.05e+00 4.52e+06 -1.0 2.09e+00 - 3.14e-04
2.11e-02f 1 5 0
4 5.9284262e+03 6.43e-02 2.23e+06 -1.0 7.64e-01 - 5.41e-04
1.00e+00f 1 5 0
5 5.5537365e+03 1.29e-02 1.25e+06 -1.0 5.96e-01 4.4 4.42e-01
1.00e+00f 1 7 0
6 5.5260089e+03 7.57e-05 1.59e+05 -1.0 1.14e-01 - 8.72e-01
1.00e+00f 1 7 0
7 5.5171774e+03 3.73e-05 1.59e+03 -1.0 1.47e-02 3.9 9.90e-01
1.00e+00f 1 8 0
8 5.4937120e+03 2.40e-04 1.22e+02 -1.0 4.10e-02 3.5 9.90e-01
1.00e+00f 1 9 0
9 5.4416411e+03 1.57e-03 1.03e+02 -1.0 1.04e-01 3.0 9.90e-01
1.00e+00f 1 10 0
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du
alpha_pr ls InReg SiReg
10 5.3638043e+03 9.07e-03 2.99e+06 -3.8 2.08e-01 2.5 6.97e-01
1.00e+00f 1 11 0
11 5.3285191e+03 1.24e-02 1.22e+06 -3.8 2.65e-01 2.0 5.92e-01
4.86e-01f 1 12 0
12 5.3277064e+03 1.22e-02 6.92e+05 -3.8 2.82e-01 1.6 4.33e-01
1.57e-02f 1 13 0
13 5.3219843e+03 1.14e-02 2.17e+05 -3.8 6.58e-01 1.1 6.86e-01
9.38e-02f 1 14 0
14 5.3083730e+03 7.55e-03 1.46e+04 -3.8 3.11e-01 - 9.33e-01
3.38e-01f 1 14 0
15 5.2971656e+03 3.89e-04 1.96e+02 -3.8 2.25e-01 - 1.00e+00
1.00e+00f 1 14 0
16 5.2974069e+03 3.63e-07 2.79e-02 -3.8 1.51e-02 - 1.00e+00
1.00e+00h 1 14 0
17 5.2974072e+03 4.21e-11 4.23e-08 -3.8 1.60e-05 - 1.00e+00
1.00e+00h 1 14 0
18 5.2974067e+03 1.29e-09 1.64e-06 -5.0 8.42e-05 - 1.00e+00
1.00e+00f 1 14 0
Number of Iterations....: 18
(scaled) (unscaled)
Objective...............: 5.2974067359191949e+03 5.2974067359191949e+03
Dual infeasibility......: 1.6380770751915933e-06 1.6380770751915933e-06
Constraint violation....: 1.2912881042215218e-09 1.2912881042215218e-09
Complementarity.........: 1.0003508439642409e-05 1.0003508439642409e-05
Overall NLP error.......: 1.0003508439642409e-05 1.0003508439642409e-05
Number of objective function evaluations = 19
Number of objective gradient evaluations = 19
Number of equality constraint evaluations = 19
Number of inequality constraint evaluations = 19
Number of equality constraint Jacobian evaluations = 19
Number of inequality constraint Jacobian evaluations = 19
Number of Lagrangian Hessian evaluations = 18
Total CPU secs in IPOPT (w/o function evaluations) = 0.005
Total CPU secs in NLP function evaluations = 0.000
EXIT: Optimal Solution Found.
List of user-set options:
Name Value used
bound_frac = 0.0001 yes
bound_push = 0.0001 yes
limited_memory_max_history = 6 yes
linear_solver = ma27 yes
max_iter = 100 yes
mu_init = 1e-05 yes
mu_strategy = monotone yes
nlp_scaling_method = none yes
output_file = ipopt.out yes
print_level = 5 yes
print_user_options = yes yes
structured_bfgs_linearalg = yes yes
tol = 0.0001 yes
use_cholfact_lowranksol = no yes
warm_start_bound_frac = 1e-16 yes
warm_start_bound_push = 1e-16 yes
warm_start_init_point = yes yes
warm_start_mult_bound_push = 1e-16 yes
warm_start_slack_bound_frac = 1e-16 yes
warm_start_slack_bound_push = 1e-16 yes
This is Ipopt version 3.12.0, running with linear solver ma27.
Number of nonzeros in equality constraint Jacobian...: 164
Number of nonzeros in inequality constraint Jacobian.: 139
Number of nonzeros in Lagrangian Hessian.............: 252
Total number of variables............................: 23
variables with only lower bounds: 0
variables with lower and upper bounds: 23
variables with only upper bounds: 0
Total number of equality constraints.................: 18
Total number of inequality constraints...............: 21
inequality constraints with only lower bounds: 0
inequality constraints with lower and upper bounds: 21
inequality constraints with only upper bounds: 0
iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du
alpha_pr ls InReg SiReg
0 5.2974067e+03 1.29e-09 1.64e-06 -5.0 0.00e+00 - 0.00e+00
0.00e+00 0 14 0
Number of Iterations....: 0
(scaled) (unscaled)
Objective...............: 5.2974067359191949e+03 5.2974067359191949e+03
Dual infeasibility......: 1.6380770751915933e-06 1.6380770751915933e-06
Constraint violation....: 1.2912881042215218e-09 1.2912881042215218e-09
Complementarity.........: 1.0003508439642409e-05 1.0003508439642409e-05
Overall NLP error.......: 1.0003508439642409e-05 1.0003508439642409e-05
Number of objective function evaluations = 1
Number of objective gradient evaluations = 1
Number of equality constraint evaluations = 1
Number of inequality constraint evaluations = 1
Number of equality constraint Jacobian evaluations = 1
Number of inequality constraint Jacobian evaluations = 1
Number of Lagrangian Hessian evaluations = 0
Total CPU secs in IPOPT (w/o function evaluations) = 0.000
Total CPU secs in NLP function evaluations = 0.000
EXIT: Optimal Solution Found.
On May 7, 2016 8:37 AM, "Илья Палачев" <iliyapalachev at gmail.com> wrote:
> Hello,
>
> I'm having the question regarding the similar problem.
>
> While trying to do the following:
> 1. Solve some problem (it succeeds in ~2 minutes)
> ... and save primal and dual variables in some intermediate buffers using
> finalize_solution callback.
> 2. Re-run the solution of the same problem (it succeeds almost in the same
> time)
>
> But:
> The last iteration of the 1st run shows the following line in the log:
>
> iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du
> alpha_pr ls
> 64 1.5248822e-04 0.00e+00 1.01e-16 -11.0 5.36e-02 - 1.00e+00
> 1.00e+00h 1
> But the first iteration of the 2nd run shows the following:
>
> iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du
> alpha_pr ls
> 0 1.5478317e-04 0.00e+00 1.00e-03 -1.0 0.00e+00 - 0.00e+00
> 0.00e+00 0
>
> As you can see, dual infeasibility (inf_du) is different at these two
> iterations. Is it correct?
>
> It's very likely that the difference between them (~ 0.001) is some
> internal constant, such as other constants that are described in the
> options manual that are also having this default value (0.001). Is there
> any option that controls that parameter?
>
> Thanks in advance,
> Ilya
>
> 06.11.2015, 22:24, "Stefan Vigerske" <stefan at math.hu-berlin.de>:
> > Hi,
> >
> > as an interior point solver, it cannot do very much with the solution of
> > a previous problem, but the log should at least differ a bit if you set
> > a warm_start_init_point.
> > If you set this option, I believe init_z and init_lambda should be true
> > in TNLP::get_starting_point(). You might want to verify this.
> >
> > Stefan
> >
> > On 11/05/2015 11:40 AM, max leo correa cordova wrote:
> >> Hello. I am using IPOPT for solving MPC using the c++ interface. I
> want to improve the computational time in the solution by using the final
> values of primal and dual variables (x, lambda, z_l and z_u) in the
> previous prediction. I have read the documentation and used:
> >>
> >> app->Options()->SetStringValue("warm_start_init_point","yes");
> >>
> >> and initialize lambda, z_l and z_u in MyNLP::get_starting_point(), but
> the results are the same and seems that Ipopt doesn't use the dual initial
> values. Could you help me please how to do that correct or is it factible
> to use this approach of warm starting? I will apprecciate your help.
> >>
> >> Thanks in advance
> >> Max
> >>
> >> _______________________________________________
> >> Ipopt mailing list
> >> Ipopt at list.coin-or.org
> >> http://list.coin-or.org/mailman/listinfo/ipopt
> >
> > --
> > http://www.gams.com/~stefan
> > _______________________________________________
> > Ipopt mailing list
> > Ipopt at list.coin-or.org
> > http://list.coin-or.org/mailman/listinfo/ipopt
> _______________________________________________
> Ipopt mailing list
> Ipopt at list.coin-or.org
> http://list.coin-or.org/mailman/listinfo/ipopt
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://list.coin-or.org/pipermail/ipopt/attachments/20160507/850a0c3e/attachment-0001.html>
More information about the Ipopt
mailing list