# [Ipopt] How to warm start dual variables in IPOPT?

Naiyuan Chiang sorakid507 at gmail.com
Sat May 7 09:57:32 EDT 2016

```I think the key points are:

A) switch off scaling function:

nlp_scaling_method none

(The default way evaluates the scaling factor based on the initial

B) Telling Ipopt to do warm start before the second solve:

warm_start_init_point = yes
warm_start_bound_frac = 1e-16
warm_start_bound_push = 1e-16
warm_start_mult_bound_push = 1e-16
warm_start_slack_bound_frac = 1e-16
warm_start_slack_bound_push = 1e-16

C) Use exact-hessian to solve the problem. If L-bfgs is applied, different
hessian approximation would be used in the 2nd run. This is due to that
L-bfgs uses historic information in the 1st solve.

I attached two results below and you can see that the second run has exact
same result as the solution at the first run.

Cheers,

Nai-Yuan

Senior Scientist
United Technologies Research Center
CT, USA

List of user-set options:

Name   Value                used

bound_frac = 0.0001                yes

bound_push = 0.0001                yes

limited_memory_max_history = 6                      no

linear_solver = ma27                  yes

max_iter = 100                   yes

mu_init = 0.1                   yes

mu_strategy = monotone              yes

nlp_scaling_method = none                  yes

output_file = ipopt.out             yes

print_level = 5                     yes

print_user_options = yes                   yes

structured_bfgs_linearalg = yes                    no

tol = 0.0001                yes

use_cholfact_lowranksol = no                     no

******************************************************************************

This program contains Ipopt, a library for large-scale nonlinear
optimization.

Ipopt is released as open source code under the Eclipse Public License
(EPL).

******************************************************************************

This is Ipopt version 3.12.0, running with linear solver ma27.

Number of nonzeros in equality constraint Jacobian...:      164

Number of nonzeros in inequality constraint Jacobian.:      139

Number of nonzeros in Lagrangian Hessian.............:      252

Total number of variables............................:       23

variables with only lower bounds:        0

variables with lower and upper bounds:       23

variables with only upper bounds:        0

Total number of equality constraints.................:       18

Total number of inequality constraints...............:       21

inequality constraints with only lower bounds:        0

inequality constraints with lower and upper bounds:       21

inequality constraints with only upper bounds:        0

iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du
alpha_pr  ls  InReg  SiReg

0  8.3631250e+03 1.55e+00 3.53e+03  -1.0 0.00e+00    -  0.00e+00
0.00e+00   0   0    0

1  7.3211706e+03 1.08e+00 4.72e+06  -1.0 4.59e-01    -  7.74e-01
2.98e-01f  1   0    0

2  7.3108635e+03 1.07e+00 4.67e+06  -1.0 1.23e+01   4.0 4.16e-04
7.24e-03f  1   5    0

3  7.2686586e+03 1.05e+00 4.52e+06  -1.0 2.09e+00    -  3.14e-04
2.11e-02f  1   5    0

4  5.9284262e+03 6.43e-02 2.23e+06  -1.0 7.64e-01    -  5.41e-04
1.00e+00f  1   5    0

5  5.5537365e+03 1.29e-02 1.25e+06  -1.0 5.96e-01   4.4 4.42e-01
1.00e+00f  1   7    0

6  5.5260089e+03 7.57e-05 1.59e+05  -1.0 1.14e-01    -  8.72e-01
1.00e+00f  1   7    0

7  5.5171774e+03 3.73e-05 1.59e+03  -1.0 1.47e-02   3.9 9.90e-01
1.00e+00f  1   8    0

8  5.4937120e+03 2.40e-04 1.22e+02  -1.0 4.10e-02   3.5 9.90e-01
1.00e+00f  1   9    0

9  5.4416411e+03 1.57e-03 1.03e+02  -1.0 1.04e-01   3.0 9.90e-01
1.00e+00f  1  10    0

iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du
alpha_pr  ls  InReg  SiReg

10  5.3638043e+03 9.07e-03 2.99e+06  -3.8 2.08e-01   2.5 6.97e-01
1.00e+00f  1  11    0

11  5.3285191e+03 1.24e-02 1.22e+06  -3.8 2.65e-01   2.0 5.92e-01
4.86e-01f  1  12    0

12  5.3277064e+03 1.22e-02 6.92e+05  -3.8 2.82e-01   1.6 4.33e-01
1.57e-02f  1  13    0

13  5.3219843e+03 1.14e-02 2.17e+05  -3.8 6.58e-01   1.1 6.86e-01
9.38e-02f  1  14    0

14  5.3083730e+03 7.55e-03 1.46e+04  -3.8 3.11e-01    -  9.33e-01
3.38e-01f  1  14    0

15  5.2971656e+03 3.89e-04 1.96e+02  -3.8 2.25e-01    -  1.00e+00
1.00e+00f  1  14    0

16  5.2974069e+03 3.63e-07 2.79e-02  -3.8 1.51e-02    -  1.00e+00
1.00e+00h  1  14    0

17  5.2974072e+03 4.21e-11 4.23e-08  -3.8 1.60e-05    -  1.00e+00
1.00e+00h  1  14    0

18  5.2974067e+03 1.29e-09 1.64e-06  -5.0 8.42e-05    -  1.00e+00
1.00e+00f  1  14    0

Number of Iterations....: 18

(scaled)                 (unscaled)

Objective...............:   5.2974067359191949e+03    5.2974067359191949e+03

Dual infeasibility......:   1.6380770751915933e-06    1.6380770751915933e-06

Constraint violation....:   1.2912881042215218e-09    1.2912881042215218e-09

Complementarity.........:   1.0003508439642409e-05    1.0003508439642409e-05

Overall NLP error.......:   1.0003508439642409e-05    1.0003508439642409e-05

Number of objective function evaluations             = 19

Number of objective gradient evaluations             = 19

Number of equality constraint evaluations            = 19

Number of inequality constraint evaluations          = 19

Number of equality constraint Jacobian evaluations   = 19

Number of inequality constraint Jacobian evaluations = 19

Number of Lagrangian Hessian evaluations             = 18

Total CPU secs in IPOPT (w/o function evaluations)   =      0.005

Total CPU secs in NLP function evaluations           =      0.000

EXIT: Optimal Solution Found.

List of user-set options:

Name   Value                used

bound_frac = 0.0001                yes

bound_push = 0.0001                yes

limited_memory_max_history = 6                     yes

linear_solver = ma27                  yes

max_iter = 100                   yes

mu_init = 1e-05                 yes

mu_strategy = monotone              yes

nlp_scaling_method = none                  yes

output_file = ipopt.out             yes

print_level = 5                     yes

print_user_options = yes                   yes

structured_bfgs_linearalg = yes                   yes

tol = 0.0001                yes

use_cholfact_lowranksol = no                    yes

warm_start_bound_frac = 1e-16                 yes

warm_start_bound_push = 1e-16                 yes

warm_start_init_point = yes                   yes

warm_start_mult_bound_push = 1e-16                 yes

warm_start_slack_bound_frac = 1e-16                 yes

warm_start_slack_bound_push = 1e-16                 yes

This is Ipopt version 3.12.0, running with linear solver ma27.

Number of nonzeros in equality constraint Jacobian...:      164

Number of nonzeros in inequality constraint Jacobian.:      139

Number of nonzeros in Lagrangian Hessian.............:      252

Total number of variables............................:       23

variables with only lower bounds:        0

variables with lower and upper bounds:       23

variables with only upper bounds:        0

Total number of equality constraints.................:       18

Total number of inequality constraints...............:       21

inequality constraints with only lower bounds:        0

inequality constraints with lower and upper bounds:       21

inequality constraints with only upper bounds:        0

iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du
alpha_pr  ls  InReg  SiReg

0  5.2974067e+03 1.29e-09 1.64e-06  -5.0 0.00e+00    -  0.00e+00
0.00e+00   0  14    0

Number of Iterations....: 0

(scaled)                 (unscaled)

Objective...............:   5.2974067359191949e+03    5.2974067359191949e+03

Dual infeasibility......:   1.6380770751915933e-06    1.6380770751915933e-06

Constraint violation....:   1.2912881042215218e-09    1.2912881042215218e-09

Complementarity.........:   1.0003508439642409e-05    1.0003508439642409e-05

Overall NLP error.......:   1.0003508439642409e-05    1.0003508439642409e-05

Number of objective function evaluations             = 1

Number of objective gradient evaluations             = 1

Number of equality constraint evaluations            = 1

Number of inequality constraint evaluations          = 1

Number of equality constraint Jacobian evaluations   = 1

Number of inequality constraint Jacobian evaluations = 1

Number of Lagrangian Hessian evaluations             = 0

Total CPU secs in IPOPT (w/o function evaluations)   =      0.000

Total CPU secs in NLP function evaluations           =      0.000

EXIT: Optimal Solution Found.
On May 7, 2016 8:37 AM, "Илья Палачев" <iliyapalachev at gmail.com> wrote:

> Hello,
>
> I'm having the question regarding the similar problem.
>
> While trying to do the following:
> 1. Solve some problem (it succeeds in ~2 minutes)
> ... and save primal and dual variables in some intermediate buffers using
> finalize_solution callback.
> 2. Re-run the solution of the same problem (it succeeds almost in the same
> time)
>
> But:
> The last iteration of the 1st run shows the following line in the log:
>
> iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du
> alpha_pr  ls
>   64  1.5248822e-04 0.00e+00 1.01e-16 -11.0 5.36e-02    -  1.00e+00
> 1.00e+00h  1
> But the first iteration of the 2nd run shows the following:
>
> iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du
> alpha_pr  ls
>    0  1.5478317e-04 0.00e+00 1.00e-03  -1.0 0.00e+00    -  0.00e+00
> 0.00e+00   0
>
> As you can see, dual infeasibility (inf_du) is different at these two
> iterations. Is it correct?
>
> It's very likely that the difference between them (~ 0.001) is some
> internal constant, such as other constants that are described in the
> options manual that are also having this default value (0.001). Is there
> any option that controls that parameter?
>
> Ilya
>
> 06.11.2015, 22:24, "Stefan Vigerske" <stefan at math.hu-berlin.de>:
> > Hi,
> >
> > as an interior point solver, it cannot do very much with the solution of
> > a previous problem, but the log should at least differ a bit if you set
> > a warm_start_init_point.
> > If you set this option, I believe init_z and init_lambda should be true
> > in TNLP::get_starting_point(). You might want to verify this.
> >
> > Stefan
> >
> > On 11/05/2015 11:40 AM, max leo correa cordova wrote:
> >>  Hello. I am using IPOPT for solving MPC using the c++ interface. I
> want to improve the computational time in the solution by using the final
> values of primal and dual variables (x, lambda, z_l and z_u) in the
> previous prediction. I have read the documentation and used:
> >>
> >>  app->Options()->SetStringValue("warm_start_init_point","yes");
> >>
> >>  and initialize lambda, z_l and z_u in MyNLP::get_starting_point(), but
> the results are the same and seems that Ipopt doesn't use the dual initial
> values. Could you help me please how to do that correct or is it factible
> to use this approach of warm starting? I will apprecciate your help.
> >>
> >>  Max
> >>
> >>  _______________________________________________
> >>  Ipopt mailing list
> >>  Ipopt at list.coin-or.org
> >>  http://list.coin-or.org/mailman/listinfo/ipopt
> >
> > --
> > http://www.gams.com/~stefan
> > _______________________________________________
> > Ipopt mailing list
> > Ipopt at list.coin-or.org
> > http://list.coin-or.org/mailman/listinfo/ipopt
> _______________________________________________
> Ipopt mailing list
> Ipopt at list.coin-or.org
> http://list.coin-or.org/mailman/listinfo/ipopt
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://list.coin-or.org/pipermail/ipopt/attachments/20160507/850a0c3e/attachment-0001.html>
```