[Coin-ipopt] Problem in memory allocation

[Andreas Waechter]

Hi Ragini,

Looking at the output giving the problems size, it says that you have 5952
variables and 496 constraints.  It also says that the number of nonzeros
in the constraint Jacobian is 2952192 = 5952 x 496, i.e. you are telling
Ipopt that the constraint Jacobian is dense.

Is that really correct?  Does every single constraint in your problem
depend on every single variable, ie., is the partial derivative of each
constraint with respect to each variable nonzero for some values of the
variables?

If not, then you should rewrite your function for computing the constraint
Jacobian to really include only those elements that can be nonzero at some
point.  Exploiting sparsity is essential when solving large-scale
problems.

If yes, then I'm not surprise about the failure, and I don't think there
is much we can do at this point.  Ipopt has been written under the
assumption that the problems (or more specifically constraint Jacobians,
and possibly Lagrangian Hessians) are sparse, and it uses MA27 to solve
the linear systems.  MA27 is a sparse linear solver, and using it to solve
dense problems is not a good idea.  On the other hand, maybe it is
possible to reformulate your optimization problem (possibly by introducing
auxilliary variables) so that it maybe has more variables and constraints,
but so that the Jacobian matrix is sparse.

I hope this helps,

Andreas


On Mon, 4 Apr 2005, Verma, Ragini wrote:

> I am using Ipopt on Linux  with a c++ interface.
>
> In my problem, I have 12*M variables and M equality constraints and  bound
> constraints on all of the variables ie 12M of them. The objective function
> is non-linear. The constraints are linear.
>
> While the algo performs well for M = 100 and lower, for M = 500 it starts
> giving memory allocation errors. I am including one such error message:
>
>
> ****************************************************************************
> **
> This program contains IPOPT, a program for large-scale nonlinear
> optimization.
>    IPOPT is released as open source under the Common Public License (CPL).
>                For more information visit www.coin-or.org/Ipopt
> ****************************************************************************
> **
>
> Number of variables           :     5952
>    of which are fixed         :        0
> Number of constraints         :      496
> Number of lower bounds        :     5952
> Number of upper bounds        :     5952
> Number of nonzeros in Jacobian:  2952192
>
> ITER     ERR       MU      ||C||   ||YPY||  ||PZ||     ||D||   ALFA(V)
> ALFA(X)     NU   #LS       F      #cor   Regu    CPU(s)
>     0 .100E+10c .100E+00 .247E-14 .000E+00 .000E+00  .000E+00 .000E+00
> .000E+00  .000E+00  0 0.14080873E+08  0 .000E+00 .253E+04
>  ma27_call:  LIWMA increased from 29613690 to 14825440
>  ma27_call:  LA increased from 2958640 to 14825440
>  ma27_call:  LIWMA increased from 14825440 to 74574845
>  ma27_call:  LA increased from 14825440 to 74574845
>  ma27_call:  LIWMA increased from 74574845 to 373470645
>  ma27_call:  LA increased from 74574845 to 373470645
>  ma27_call:  LIWMA increased from 373470645 to 1867949645
>  ma27_call:  LA increased from 373470645 to 1867949645
>  get_step_full: IP_MALLOC returned  -1 for KKT (realloc)
>  solve_barrier: get_step_full returns IERR =  96
>  mainloop: Error: solve_barrier ends with IERR =  96
>
> Number of iterations taken .............                      0
> Final value of objective function is.... 0.1408087347920045E+08
>
> Errors at final point                      (scaled)       (unscaled)
> Final maximal constraint violation is... 0.111022E-15    0.111022E-15
> Final value for dual infeasibility is... 0.100000E+03    0.128967E+07
> Final value of complementarity error is. 0.100000E+10    0.100000E+10
>
> The objective function was evaluated      1 times.
> The constraints were evaluated            1 times.
>
> EXIT: Error during dynamic memory allocation
>
> CPU seconds spent in IPOPT and function evaluations =       2542.4099
>
>
>
>
> M = 500 is a lower end of my problem, I am looking at M = 5000 or more in my
> real problem. I was wondering whether it will be feasible to use Ipopt with
> such a large size problem and how to solve the memory management problem.
>
> Thanks,
> .
> Ragini
>
> ---------------------------------------
> Ragini Verma, Ph.D.
> Section of Biomedical Image Analysis
> Department of Radiology
> University of Pennsylvania
> Add : 3600 Market St. Suite 380
>       Philadelphia, PA 19104
> Tel : 215 662 7471
> Fax : 215 614 0266
> Email : Ragini.Verma at uphs.upenn.edu
>
>