[Ipopt] ipopt with l-bfgs: slow progress on unbounded linear objective

Sat Jun 20 09:16:54 EDT 2015

Hi,

On 06/20/2015 01:36 PM, Ipopt User wrote:
> Interesting. I have modified and attached Ipopt/examples/Cpp_example/ to
> demonstrate the issue.
>
> Not sure why, but app->Options()->SetNumericValue("print_level", 8); does
> not generate extra output, so I cannot directly compare our log files.

You need to use SetIntegerValue(). There is a message in the log that 
says that SetNumericValue was not working.

> THe
> output below is for Ipopt 3.12.3, but I was able to reproduce this issue
> with Ipopt 3.11.

Yes, it should be the same.

The difference seems to come because in your implementation, Ipopt does 
not know that the x variable is linear, so it considers it as nonlinear.
If you implement IpTNLP::get_number_of_nonlinear_variables() and 
Ipopt::get_list_of_nonlinear_variables(), then you get similar output as 
I do. Attached the modified code.

This essentially disables the Hessian approximation again, though. There 
is a message "Hessian approximation will be done in smaller space of 
dimension 0 (instead of 1)" in the log when the print level is high enough.

Stefan

>
> ~/IPOPT/build/Ipopt/examples/Cpp_example$ ./cpp_example | head -n 100
>
> ******************************************************************************
> This program contains Ipopt, a library for large-scale nonlinear
> optimization.
>   Ipopt is released as open source code under the Eclipse Public License
> (EPL).
>           For more information visit http://projects.coin-or.org/Ipopt
> ******************************************************************************
>
> This is Ipopt version 3.12.3, running with linear solver ma27.
>
> Number of nonzeros in equality constraint Jacobian...:        0
> Number of nonzeros in inequality constraint Jacobian.:        0
> Number of nonzeros in Lagrangian Hessian.............:        0
>
> Total number of variables............................:        1
>                       variables with only lower bounds:        1
>                  variables with lower and upper bounds:        0
>                       variables with only upper bounds:        0
> Total number of equality constraints.................:        0
> Total number of inequality constraints...............:        0
>          inequality constraints with only lower bounds:        0
>     inequality constraints with lower and upper bounds:        0
>          inequality constraints with only upper bounds:        0
>
> iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du
> alpha_pr  ls
>     0 -1.0000000e+00 0.00e+00 2.00e+00   0.0 0.00e+00    -  0.00e+00
> 0.00e+00   0
>     1 -1.0933387e+01 0.00e+00 1.01e+00   1.0 9.93e+00    -  9.84e-01
> 1.00e+00f  1
>     2 -2.0934581e+01 0.00e+00 1.00e+00  -1.0 1.00e+01    -  1.00e+00
> 1.00e+00f  1
>     3 -3.0934593e+01 0.00e+00 1.00e+00  -2.9 1.00e+01    -  1.00e+00
> 1.00e+00f  1
>     4 -4.0934594e+01 0.00e+00 1.00e+00  -4.9 1.00e+01    -  1.00e+00
> 1.00e+00f  1
>
>
> On Sat, Jun 20, 2015 at 6:53 AM, Stefan Vigerske <stefan at math.hu-berlin.de>
> wrote:
>
>> Hi,
>>
>> I cannot reproduce this behavior (and I assume you mean that x is
>> increasing only slowly, not decreasing).
>> Even when putting x>=0 as a constraint instead of just a variable bound,
>> Ipopt reports unboundedness after 35 iterations.
>>
>> The values for x in all iterations are
>> curr_x[    1]{x}= 0.0000000000000000e+00
>> curr_x[    1]{x}= 1.0000000000000000e+04
>> curr_x[    1]{x}= 4.0000000000000000e+04
>> curr_x[    1]{x}= 1.3000000000000000e+05
>> curr_x[    1]{x}= 4.0000000000000000e+05
>> curr_x[    1]{x}= 1.2100000000000000e+06
>> curr_x[    1]{x}= 3.6400000000000005e+06
>> curr_x[    1]{x}= 1.0930000000000002e+07
>> curr_x[    1]{x}= 3.2800000000000007e+07
>> curr_x[    1]{x}= 9.8410000000000030e+07
>> curr_x[    1]{x}= 2.9524000000000012e+08
>> curr_x[    1]{x}= 8.8573000000000036e+08
>> curr_x[    1]{x}= 2.6572000000000010e+09
>> curr_x[    1]{x}= 7.9716100000000029e+09
>> curr_x[    1]{x}= 2.3914840000000011e+10
>> curr_x[    1]{x}= 7.1744530000000046e+10
>> curr_x[    1]{x}= 2.1523360000000012e+11
>> curr_x[    1]{x}= 6.4570081000000037e+11
>> curr_x[    1]{x}= 1.9371024400000010e+12
>> curr_x[    1]{x}= 5.8113073300000039e+12
>> curr_x[    1]{x}= 1.7433922000000012e+13
>> curr_x[    1]{x}= 5.2301766010000031e+13
>> curr_x[    1]{x}= 1.5690529804000012e+14
>> curr_x[    1]{x}= 4.7071589413000044e+14
>> curr_x[    1]{x}= 1.4121476824000012e+15
>> curr_x[    1]{x}= 4.2364430472100040e+15
>> curr_x[    1]{x}= 1.2709329141640014e+16
>> curr_x[    1]{x}= 3.8127987424930048e+16
>> curr_x[    1]{x}= 1.1438396227480014e+17
>> curr_x[    1]{x}= 3.4315188682441050e+17
>> curr_x[    1]{x}= 1.0294556604732416e+18
>> curr_x[    1]{x}= 3.0883669814197350e+18
>> curr_x[    1]{x}= 9.2651009442592154e+18
>> curr_x[    1]{x}= 2.7795302832777658e+19
>> curr_x[    1]{x}= 8.3385908498332975e+19
>> curr_x[    1]{x}= 1.8338590849833298e+20
>>
>> I attach the full log.
>>
>> If I define x>=0 as variable bound, then Ipopt needs only 6 iterations.
>>
>> Stefan
>>
>>
>>
>> On 06/18/2015 05:49 PM, Ipopt User wrote:
>>
>>> When I try to solve the simple scalar problem min{-x : x >= 0} with IPOPT
>>> and use hessian_approximation = limited-memory, the progress is very slow.
>>> Essentially, x is reduced by 1 in each iteration.
>>>
>>>   From what I understand, this is due to the Hessian approximation, which
>>> in
>>> this case is just the identity matrix (plus 1/x due to the log barrier
>>> term, which is significant only in the first few iterations). Solving Hs =
>>> -g, with H the Hessian, s the step and g the gradient, it finds s = 1.
>>> Indeed when I set limited_memory_init_val = 0.1, x is reduced by 10 in
>>> each
>>> iteration.
>>>
>>> This seems undesirable to me. Other L-BFGS based solvers like SNOPT and
>>> L-BFGS-B figure out that the problem is unbounded in a few iterations.
>>>
>>>
>>>
>>> _______________________________________________
>>> Ipopt mailing list
>>> Ipopt at list.coin-or.org
>>> http://list.coin-or.org/mailman/listinfo/ipopt
>>>
>>>
>>
>
>
>
> _______________________________________________
> Ipopt mailing list
> Ipopt at list.coin-or.org
> http://list.coin-or.org/mailman/listinfo/ipopt
>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: cpp_example.cpp
Type: text/x-c++src
Size: 1937 bytes
Desc: not available
URL: <http://list.coin-or.org/pipermail/ipopt/attachments/20150620/46a4445e/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: MyNLP.cpp
Type: text/x-c++src
Size: 3359 bytes
Desc: not available
URL: <http://list.coin-or.org/pipermail/ipopt/attachments/20150620/46a4445e/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: MyNLP.hpp
Type: text/x-c++hdr
Size: 4185 bytes
Desc: not available
URL: <http://list.coin-or.org/pipermail/ipopt/attachments/20150620/46a4445e/attachment-0002.bin>