[Coin-ipopt] Re: IPOPT 3 Memory question

[Carl Laird]

I just have one more thing to add concerning the "MA27 requires more
memory messages" that might help some users.

As Andreas indicated, Ipopt has to guess how much memory will be
required for the factorization of the linear system. If the amount of
memory used by the original linear system is (Z), then Ipopt reserves
(F x Z) for the factorized linear system (where F is determined by the
options Andreas described).

When MA27 requires more memory, that means that the difference in
memory requirement between the original system and the factored system
are (to some approximation) more than F. Therefore, MA27 asking for
more memory does not mean that the system is "just big", rather it
means that the system is producing more fill-in during the
factorization then we planned for (initially a factor of 10).

While there are problem classes that inherently induce fill-in when
the linear system is factorized, this also happens when problems are
poorly scaled or are nearly singular. Therefore, if you see that MA27
requires more memory, before buying more memory or switching to a 64
bit machine, have a close look at the problem and make sure it is, in
some sense, well-conditioned. 9 times out of 10, when I see this
message, a change in formulation makes all the difference.

Anyway, I know this isn't exactly related to the original concern
(which was a different message), however, since we hear a number of
questions regarding the MA27 message, I thought I would post my two
bits.

Cheers,

Carl.



On 7/13/07, Andreas Waechter <andreasw at watson.ibm.com> wrote:
> Hi Rhoda,
>
> It is nice to hear from you - I hope you are doing well.
>
> (I'm copying my reply to the mailing list, since your observation is
> probably seen by others as well.)
>
> > I'm running a problem that is reporting this error after it runs for a while:
> >
> > EXIT: Not enough memory.
> > error running ipopt3:
> >        termination code 11
> >
> > I'm running with 1GB of physical memory and 2GB of swap. Before I go and
> > purchase more, I'd just to confirm that this message is telling me to get
> > more physical memory as opposed to just increasing some memory allocation
> > number in the code somewhere.
> >
> > I also get these on occasion, but they don't cause IPOPT to exit:
> >
> > MA27BD returned iflag=-4 and requires more memory.
>
> Before you buy more memory, I suggest two things:
>
> 1. The problem with MA27 is that it is written in Fortran 77, and doesn't
> have dynamic memory allocation.  Therefore, Ipopt has to guess beforehand,
> how much memory MA27 requires (based on the number of nonzeros).  The
> options ma27_liw_init_factor and ma27_la_init_factor determine this
> initial guess of MA27's memory requirement.  If during the factorization
> MA27 complains that it doesn't have enough memory, the guesses are
> increased by some factor, and that factor is the value of the option
> ma27_meminc_factor.  Now, the default of that second option is "10", which
> is maybe too aggressive. I would try smaller values (say, 2 or 3) in your
> case.
>
> 2. It might also help to try a different linear solver.  Starting with
> Ipopt 3.3.1, we added support for the public domain linear solver MUMPS,
> and if you are solving large problems, that one might require less memory
> (you need a Fortran 90 compiler for MUMPS, such as GNU's gfortran - then
> you can have the Ipopt Makefiles compile it for you.).  But also other
> linear solvers (such as Pardiso and WSMP, see the Ipopt documentation),
> might be better in terms of memory requirement.  Those are commercial
> solvers, for which you get the compiled libraries, but if you use them for
> academic purposes, you can get free temporary licenses.
>
> Besides this, if you are running Ipopt on a 32bit platform, I think you
> cannot use more than 2 (or possibly 3) GB anyway for one process (I
> think... definitely not more than 4GB).
>
> I hope this helps.  If you find a way to solve your problem with one of
> those suggestions (or any other way), it would be great if you could write
> a note on
>
> https://projects.coin-or.org/Ipopt/wiki/HintsAndTricks
>
> ;-)
>
> [To all Ipopt users: If you find a useful trick that helps you to make
> better use of Ipopt, please let others know about it at that wiki page!!!
> Thanks!!!]
>
> Andreas
> _______________________________________________
> Coin-ipopt mailing list
> Coin-ipopt at list.coin-or.org
> http://list.coin-or.org/mailman/listinfo/coin-ipopt
>