[Coin-ipopt] Convergence and BFGS Hessian computation
Sylvain Miossec
sylvain.miossec at aist.go.jp
Thu Oct 26 21:44:31 EDT 2006
Hi Andreas, Hi Carl,
I finally succeeded to solve my convergence problems.
I implemented an automatic scaling which consists in minimizing the
euclidean distance between the gradient coefficients and values of one.
In other words I minimize the error between gradient coefficients and
ones. By taking the log of scaled gradient values, I obtained a
quadratic problem without constraints, which is solved easily. But with
the obtained scaling parameters, the convergence was not as fast as the
one with a hand tune scaling. And I still couldn't obtain proper
convergence.
The thing is I implemented a BFGS approximation of the Hessian. The
hessian approximation is better. And for my problem size, it is not
costly to compute. Therefore I obtain faster convergence than with
L-BFGS. And it appears that I obtain proper convergence, getting all the
tolerances as small as desired.
You can look at the attached file and compare with the previous runs I
posted earlier.
I gave the following tolerances :
-tol: 1e-3
-acceptable_tol: 1e-1
-constr_viol_tol: 1e-6
-acceptable_constr_viol_tol: 1e-4
-dual_inf_tol: 1e-4
-acceptable_dual_inf_tol: 1e-2
-compl_inf_tol: 1e-5
-acceptable_compl_inf_tol: 1e-3
For the practical implementation, I programmed the BFGS computation in
the Eval_h function. And I stored the gradients of objective and
constraints computed in Eval_jac_f and Eval_jacg in order to use them in
the Eval_h function. It is not clean, but it worked.
Do you think it is because I have a special problem that I couldn't get
proper convergence with limited memory BFGS ? Or is there another reason ?
Sylvain
Andreas Waechter wrote:
> Hi Sylvain,
> From what you write it seems that you are using numerical computations
> to compute your constraint values and/or derivatives. This can easily
> explain why the primal infeasibility has so much trouble going down.
> (When you scale down your constraints by 1e-3, as you did, it of
> course also makes the constraint violation smaller by that quantity -
> max gradient 300 is fine, 0.3 might be a bit small.)
>
> AMPL would allow you to write down NLPs, if you have them in
> analytical form, i.e., all functions given as equations. You can
> write is down pretty much as you would have it on paper, so it is easy
> to model. And it would take take of evaluating the deriatives for
> you. However, if you are using numerical prodecures to compute
> constraint function values, this is difficult.
>
> In some cases one can avoid numerical procedures like integration, by
> performing the discretization in the NLP itself (some people call this
> the "simultaneous approach"). This increases the number of variables,
> but the problems are usually very sparse, and converge nicer.
>
> By the way, I also discovered a bug in my implementation of the L-BFGS
> approximation that I fixed now and put into the 3.2 stable release of
> Ipopt. You can get it via subversion from the URL
>
> https://projects.coin-or.org/svn/Ipopt/stable/3.2
>
> This might make the convergence a bit better. However, the problem of
> slowly converging dual infeasibility is still a real problem, and at
> this point I only have some simple heuristics to offer in the code...
>
> Regards,
>
> Andreas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: optim_result_bfgs2_autoscale_objscale1e-6-10_gradreduc10.zip
Type: application/x-zip-compressed
Size: 11805 bytes
Desc: not available
Url : http://list.coin-or.org/pipermail/ipopt/attachments/20061027/b973840f/attachment.bin
More information about the Coin-ipopt
mailing list