[Coin-ipopt] Convergence and BFGS Hessian computation

[Sylvain Miossec]

Hi Andreas, Hi Carl,

I finally succeeded to solve my convergence problems.

I implemented an automatic scaling which consists in minimizing the
euclidean distance between the gradient coefficients and values of one.
In other words I minimize the error between gradient coefficients and
ones. By taking the log of scaled gradient values, I obtained a
quadratic problem without constraints, which is solved easily. But with
the obtained scaling parameters, the convergence was not as fast as the
one with a hand tune scaling. And I still couldn't obtain proper
convergence.

The thing is I implemented a BFGS approximation of the Hessian. The
hessian approximation is better. And for my problem size, it is not
costly to compute. Therefore I obtain faster convergence than with
L-BFGS. And it appears that I obtain proper convergence, getting all the
tolerances as small as desired.
You can look at the attached file and compare with the previous runs I
posted earlier.
I gave the following tolerances :
-tol: 1e-3
-acceptable_tol:  1e-1
-constr_viol_tol:  1e-6
-acceptable_constr_viol_tol:  1e-4
-dual_inf_tol: 1e-4
-acceptable_dual_inf_tol: 1e-2
-compl_inf_tol: 1e-5
-acceptable_compl_inf_tol: 1e-3

For the practical implementation, I programmed the BFGS computation in
the Eval_h function. And I stored the gradients of objective and
constraints computed in Eval_jac_f and Eval_jacg in order to use them in
the Eval_h function. It is not clean, but it worked.

Do you think it is because I have a special problem that I couldn't get
proper convergence with limited memory BFGS ? Or is there another reason ?

Sylvain



Andreas Waechter wrote:
> Hi Sylvain,
> From what you write it seems that you are using numerical computations 
> to compute your constraint values and/or derivatives.  This can easily 
> explain why the primal infeasibility has so much trouble going down. 
> (When you scale down your constraints by 1e-3, as you did, it of 
> course also makes the constraint violation smaller by that quantity - 
> max gradient 300 is fine, 0.3 might be a bit small.)
>
> AMPL would allow you to write down NLPs, if you have them in 
> analytical form, i.e., all functions given as equations.  You can 
> write is down pretty much as you would have it on paper, so it is easy 
> to model.  And it would take take of evaluating the deriatives for 
> you.  However, if you are using numerical prodecures to compute 
> constraint function values, this is difficult.
>
> In some cases one can avoid numerical procedures like integration, by 
> performing the discretization in the NLP itself (some people call this 
> the "simultaneous approach").  This increases the number of variables, 
> but the problems are usually very sparse, and converge nicer.
>
> By the way, I also discovered a bug in my implementation of the L-BFGS 
> approximation that I fixed now and put into the 3.2 stable release of 
> Ipopt.  You can get it via subversion from the URL
>
> https://projects.coin-or.org/svn/Ipopt/stable/3.2
>
> This might make the convergence a bit better.  However, the problem of 
> slowly converging dual infeasibility is still a real problem, and at 
> this point I only have some simple heuristics to offer in the code...
>
> Regards,
>
> Andreas

-------------- next part --------------
A non-text attachment was scrubbed...
Name: optim_result_bfgs2_autoscale_objscale1e-6-10_gradreduc10.zip
Type: application/x-zip-compressed
Size: 11805 bytes
Desc: not available
Url : http://list.coin-or.org/pipermail/ipopt/attachments/20061027/b973840f/attachment.bin