[Coin-ipopt] Re: Putt-Putt, Sputter, Solved -- IPOPT tuning help?

[Andreas Waechter]

Hi Frank,

I apologize for the very delayed response.  I have been very busy with 
some other COIN stuff, but I guess I'm preaching to the choir...

I'm glad you decided to switch to the 3.1.0 version.  I think we improved 
performance and robustness somewhat from 3.0.1 to 3.1.0.

1. The delta_c parameter is used to handle rank-deficient constraint 
Jacobians (as you wrote).  In the restoration phase, the problem that 
Ipopt sees is a different one, and by its definition, the constraint 
Jacobian has aolways full rank.  This is why delta_c is always zero in the 
restoration phase.

2. In the first output you sent in your earlier message, I saw the 'LL' 
string at the end of each line.  This means that the constraint Jacobian 
is always rank deficient.  In what you wrote in the message below it seems 
that is has now changed, since "the iteration matrix breaks free of 
zero-valued eigenvalues after the first few iterations."

This could mean that the starting point is really chosen unluckily.  I 
don't understand what you mean by "I'm not providing any explicit 
variable initialization" in your first message.  If you are using AMPL, 
the variables are all set to 0, and Ipopt then puts them into the bounds 
(if 0 is not inside the bounds).  Often, zero has a special meaning, and 
it leads to a rank-deficient constraint Jacobian at the starting point. 
You don't have to give Ipopt a starting point that is feasible, but you 
should try to find a point at which the constraint Jacobian is not 
rank-deficient.  Maybe just a somewhat random initialization might help.

If the constraint Jacobian is always rank-deficient, you probably have 
constraints that are linearly dependend (as nonlinear functions).  In that 
case, removing redundant constraints should help.  At this point, Ipopt 
doesn't have a preprocessor that would try to detect something like this 
automatically.

3.

> * The restoration phase continues for about 300 iterations, at the end of 
> which we get:
>
>    RESTORATION PHASE RESULTS
> Optimal solution found!
> Optimal Objective Value = 1.5189365323352031E+003
> Number of Iterations = 348
> Step size for bound multipliers: 4.42e-007
> Bound multipliers after restoration phase too large (max=6.33e+006). Set all 
> to 1.
>
> That bit about the bound multipliers doesn't make sense to me, since just 
> above, they were printed as:
> ||curr_v_L||_inf = 1.8544836972327396e+001
> ||curr_v_U||_inf = 1.0513570111533468e+002
> <snip>
> ||delta_v_L||_inf = 5.4028411824574746e+001
> ||delta_v_U||_inf = 2.1209207829198019e+002

To say more about this I would need to see the output.  The detailed 
output in the output file is not always 100% accurate and can be 
misleading...

Let me know if any of this makes sense.  Next time, I will try to answer 
earlier....

Regards,

Andreas

On Tue, 13 Jun 2006, Frank J. Iannarilli wrote:

> Hi,
>
>
> Following up on my initial post:
>
> I'm now using IPOPT3.1.0.
>
> Here is some hopefully telling behavior that I've gleaned from scanning 
> output file contents.
>
> I've tried a slightly larger problem instance with various IPOPT option 
> settings.  This problem is indeed feasible, as verified by several 
> IPOPT-founded solutions.  But for default and many option variations, there 
> are either way too many iterations, or pre-mature termination (after hundreds 
> of iterations) to infeasible points.
>
>
> * At the outset of iterations, the iteration matrix has zero-valued 
> eigenvalues, thus delta_c is employed for conditioning. To achieve 
> convergence (tol=1e-4) in about 400-500 iterations, I needed to boost 
> delta_c:
> jacobian_regularization_value 1e-3  # delta_c  (default: 1e-8)
> jacobian_regularization_exponent 1e-1    (default: 2.5e-1)
> Otherwise, default option setting leads to infeasible pt termination. 
> Interestingly, with default value of delta_c, the iteration matrix breaks 
> free of zero-valued eigenvalues after the first few iterations.
>
>
> * (With boosted delta_c), the IPOPT filter "coughs" at about 90 iterations, 
> thereby beginning the Restoration Phase.  Before this, delta_c conditoning 
> continues to be employed; thereafter not.  Also, before this, the lagrange 
> multiplier terms and their gradients ||.||_inf start climbing to ~1e6, 1e7, 
> whereas the primal terms are around 1e1; thereafter, all the primal and dual 
> variables and gradients are comparable ~ 1e2.
>
>
>
> * The restoration phase continues for about 300 iterations, at the end of 
> which we get:
>
>    RESTORATION PHASE RESULTS
> Optimal solution found!
> Optimal Objective Value = 1.5189365323352031E+003
> Number of Iterations = 348
> Step size for bound multipliers: 4.42e-007
> Bound multipliers after restoration phase too large (max=6.33e+006). Set all 
> to 1.
>
> That bit about the bound multipliers doesn't make sense to me, since just 
> above, they were printed as:
> ||curr_v_L||_inf = 1.8544836972327396e+001
> ||curr_v_U||_inf = 1.0513570111533468e+002
> <snip>
> ||delta_v_L||_inf = 5.4028411824574746e+001
> ||delta_v_U||_inf = 2.1209207829198019e+002
>
>
>
> * Anyway, after end of restoration phase, another 200 iterations commence 
> before optimal solution is reached.
>
>
> ==== my ipopt.opt contents for achieving convergence: =====
> ma27_pivtol 1e-14   # default: 1e-8
> tol 1e-4  # > 0 default: 1e-8
> mu_strategy adaptive
> quality_function_centrality cubed-reciprocal
> quality_function_balancing_term cubic
> jacobian_regularization_value 1e-3  # delta_c
> jacobian_regularization_exponent 1e-1
> ============================================================
>
>
> * I tried various perturbations on the above theme:
>
> - pivtol made no difference; if anything, relaxing it caused infeasible pt 
> termination.
>
> - toggling the following was of no help:
> linear_scaling_on_demand
> nlp_scaling_method
> alpha_for_y # primal (default)
>
> - To perturb problem scaling, in my model I tried taking square-roots of 
> vector inner-products rather than working with raw inner products, to no 
> avail.  The IPOPT behavior was about the same.
>
>
> ---------
>
> I continue to toy with the notion of initializing the variables to feasible 
> values, but at issue is that I can't initialize them ALL to feasible values. 
> And it seems partial initialization would be pointless.
>
> Any ideas for IPOPT tuning against the aforementioned behaviors appreciated!
>
> Regards,
>
>
>
>
>
>
> Frank J. Iannarilli, franki at aerodyne.com
> Aerodyne Research, Inc., 45 Manning Rd., Billerica, MA 01821 USA
> www.aerodyne.com/cosr/cosr.html
> _______________________________________________
> Coin-ipopt mailing list
> Coin-ipopt at list.coin-or.org
> http://list.coin-or.org/mailman/listinfo/coin-ipopt
>