[Coin-ipopt] Re: EXIT: Error during dynamic memory allocation
Andreas Waechter
andreasw at watson.ibm.com
Mon Apr 25 12:13:06 EDT 2005
Hi Hans,
What about the questions I asked you about the lines
> estimated double precision work space requirement = 24078010
> estimated integer work space requirement = 37665174
?
This is not Ipopt output, and you should not have to call ESTIMATE_WS
(which is what I suspect you are doing). I suggest you check where these
lines come from and you remove a call to ESTIMATE_WS if that is what you
are doing.
Also, as I mentioned earlier, your model still seems to have almost 100
non-zero elements in each row of the Jacobian on average. This seems like
a lot, and can be responsible for the fact that you run out of memory.
All I can suggest to you is to check you code for the derivatives again.
Try to figure out why you have so many nonzeros, and whether this is
really correct (or if there are many elements that you give to Ipopt in
the list of nonzero elements but that are actually always zero)? Also, in
order to check if your derivatives are correct, you could implement a
finite difference test.
By the way, with the options that you are using now, you are not using the
reduced space option, and ma28 is not longer called, i.e. any change you
make in ma28_call does not have an effect.
Andreas
On Sun, 24 Apr 2005, hong weirong wrote:
> Hi Andreas,
>
> I have not solved my problem yet, and hope to get your help again.
> I checked some mistakes in my optimization problem written in FORTRAN,
> and chosen 4 times for the hardcoded factor in the file ma28_call.F.
> When running the executive file, the followings were appeared.
>
> ---------------------------------------------------------------------------------------------------------------------
>
> estimated double precision work space requirement = 24078010
> estimated integer work space requirement = 37665174
> Parameter DTOL has multiple assignments - using 0.10000D-05 .
> ******************************************************************************
> This program contains IPOPT, a program for large-scale nonlinear optimization.
> IPOPT is released as open source under the Common Public License (CPL).
> For more information visit www.coin-or.org/Ipopt
> ******************************************************************************
> Number of variables : 28160
> of which are fixed : 0
> Number of constraints : 28080
> Number of lower bounds : 440
> Number of upper bounds : 80
> Number of nonzeros in Jacobian: 2646081
>
> ITER ERR MU ||C|| ||D|| ALFA(X) #LS F Regu
> 0 .158D+04p .100D+00 .369D+05 .000D+00 .000D+00 0 0.14363660D+00 .000D+00
> get_step_full: IP_MALLOC returned -1 for KKT
> solve_barrier: get_step_full returns IERR = 96
> mainloop: Error: solve_barrier ends with IERR = 96
> Number of iterations taken ............. 0
> Final value of objective function is.... 0.1436366000000000D+00
> Errors at final point (scaled) (unscaled)
> Final maximal constraint violation is... 0.158330D+04 0.158330D+04
> Final value for dual infeasibility is... 0.100000D+01 0.100000D+01
> Final value of complementarity error is. 0.100708D+00 0.100708D+00
> The objective function was evaluated 1 times.
> The constraints were evaluated 1 times.
> EXIT: Error during dynamic memory allocation
> CPU seconds spent in IPOPT and function evaluations = 11.6173
> Error initializing algorithmic parameters.
> IERR = 96
> You might want to check IPOPT.DAT (if you set QCNR>0)
>
> ------------------------------------------------------------------------------------------------------------------------
> the attached file is the IPOPT.OUT.
> What's the meaning of 'Do you call Ipopt multiple times?' in your mail?
> I wonder when this situation will appear? I have not even intended to
> call Ipopt multiple times.
> My optimization problem is about the dynamic optimization of operation
> of distillation system. I build the model equations (equality
> constraints) and Jacobian matrix in Fortran by myself. From your
> analysis, maybe some mistakes exist in my program. But I dont know where
> I begin to check. Please give me some suggestion for it.
>
> In additon, I used the default options, except for
> IQUASI 6
> IPRINT 3
> IFILE 1
> IOUTPUT 1
>
>
> the same words were appeared.
>
> Best Regards,
> Weirong
>
>
>
>
>
>
>
>
> Andreas Waechter <andreasw at watson.ibm.com> wrote:
> Hi Hans,
>
> There is something weird going on. The output you sent me says that the
> number of nonzeros in the Jacobian is 2646081, and the number of variables
> are 28160 - that's not really large. However, the output also says that
> the work space arrays are larger than 200,000,000 which is a 100 times
> larger than the number of nonzeros.
>
> Actually, where are the lines
>
> estimated double precision work space requirement = 55830982
> estimated integer work space requirement = 85294632
>
> coming from? I don't think this is Ipopt output. Are you trying to
> allocate memory for Ipopt? (The way you compiled Ipopt, it allocates
> memory on its own and you don't have to do anything like that.)
>
> What options do you set? How did you pose the optimization problem
> (Fortran, AMPL?)? If it is written in Fortran, are you sure your
> interface to Ipopt and your problem functions (EVAL_F...) are correct? Do
> you call Ipopt multiple times?
>
> If you want to try to run it again, please use the default options, except
> for
>
> IQUASI 6
> IPRINT 3
> IFILE 1
> IOUTPUT 1
>
> Regards,
>
> Andreas
>
>
>
> On Sat, 5 Mar 2005, hong weirong wrote:
>
> > Hi Andreas,
> > I am sorry that I didnot introduce myself to you. My name is Weirong
> > Hong (Chinese name, do you remember?), Hans is my English name. I have
> > even studied at TU-Berlin in 2003-2004. In that time I even wrote you to
> > ask a lots of questions about IPOPT. Now I return to China and try to
> > apply IPOPT to chemical engineering optimization problem. Hope you help
> > me again.
> > I have even applied IPOPT to optimal control problem of CSTR. There
> > were 550 variables, 450 equality constraints, lower and upper bounds of
> > all variables in that problem. The results were satisfactory.
> > Let's come back to my problem. My current problem is about optimization
> > operation of distillation column. Accoding to your suggestion I used the
> > full space version of the code with the option iquasi=6 in IRIX64
> > system. 'Killed' happened again. Maybe the issue is that f77 is the
> > MIPSpro FORTRAN 77 compiler in IRIX64? Alternatively, I used the
> > reduced-space in which I modified 40 times to 10(4) times in
> > ma28_call.F, 'Killed' happened also. Finally, I returned to Linux system
> > with ./configue F77=g77. The situations were as followings.
> >
> > 1. in full-space and reduced-space option with 10 times in ma28_call.F, the following messages were displayed after input executable file:
> > -----------------------------------------------------------------------------------------------------------------------------
> >
> > hong at debian:~/ipopt-1/column$ ./example
> > estimated double precision work space requirement = 55830982
> > estimated integer work space requirement = 85294632
> > Parameter DTOL has multiple assignments - using 0.10000E-05 .
> > ******************************************************************************************
> > This program contains IPOPT, a program for large-scale nonlinear optimization.
> > IPOPT is released as open source under the Common Public License (CPL).
> > For more information visit www.coin-or.org/Ipopt
> > *****************************************************************************************
> > Number of variables : 28160
> > of which are fixed : 0
> > Number of constraints : 28080
> > Number of lower bounds : 440
> > Number of upper bounds : 80
> > Number of nonzeros in Jacobian: 2646081
> > Segmentation fault
> >
> > -----------------------------------------------------------------------------------------------------------------------------
> >
> > 2. in reduced-space option with 40 times in ma28_call.F, the following messages were displayed after input executable file:
> >
> > ----------------------------------------------------------------------------------------------------------------------------
> > estimated double precision work space requirement = 214595842
> > estimated integer work space requirement = 323441922
> > Parameter DTOL has multiple assignments - using 0.10000D-05 .
> > ******************************************************************************
> > This program contains IPOPT, a program for large-scale nonlinear optimization.
> > IPOPT is released as open source under the Common Public License (CPL).
> > For more information visit www.coin-or.org/Ipopt
> > ******************************************************************************
> > ipopt: Error allocating double precision work space: -1
> > Error initializing algorithmic parameters.
> > IERR = 96
> > You might want to check IPOPT.DAT (if you set QCNR>0)
> >
> > ----------------------------------------------------------------------------------------------------------------------------
> >
> > What's the meaning of 'Segmentation fault' ? Could you give me interpretations? The attachment is the config.log file of Linux system. Thanks.
> >
> >
> > best wishes,
> > Weirong
> >
> >
> >
> >
> >
> > Andreas Waechter wrote:
> > Hi Hans,
> >
> > Sorry for the delayed response, I was travelling.
> >
> > You chose the reduced space option of the code - I'm not so confident that
> > that will work very well for a problem that big, unless you provide a
> > good basis for the constraint Jacobian. But looking at the PARAMS.DAT
> > file you seem to already have some experience with the reduced space
> > option...
> >
> > The problem is that I'm not sure how much to trust MA28 to factorize
> > problems that big. That might simply cause numerical issues. I have to
> > admit that I haven't worked with the reduced-space option in quite a
> > while, and I know that there are things that could be improved, but
> > currently I don't have the time to look into this. If you have access to
> > MA48, you could try to see if that works better.
> >
> > I would suggest first to see what happens if you use the full space
> > version of the code with the option iquasi=-6 or 6. That might already
> > help for the memory issue, but maybe convergence will not be good.
> >
> > Alternatively, when estimating the memory requirement for MA28, Ipopt uses
> > currently a hardcoded factor in the file ma28_call.F (of 40 times the
> > number of nonzeros for the expected fill-in). This is probably much
> > larger than you need it. It might be worth trying to see if smaller
> > values avoid that memory problem. You might want to try values like 4 or
> > 10. The lines to change are 226 and 227 in
> > COIN/Ipopt/IPOPT/ipopt/ma28_call.F. If that solves the memory problem for
> > you, please let me know, and I can make this a parameter for PARAMS.DAT,
> > so that one doesn't have to recompile the code every time.
> >
> > Finally, in case your IRIX64 supports 64bit, you could try to see if you
> > can compile Ipopt to give 64bit code... (even though you have only 4GB, it
> > might still help in case the operating system allows you to allocate
> > larger chunks of memory, which are only physicaly requested by the OS when
> > memory is acctually accessed - that is I think how Linux does it.)
> >
> > Out of curiosity, what kind of application are you working on? (I'm
> > always looking for people to list on the Ipopt home page that have used
> > Ipopt successfully for interesting stuff...)
> >
> > In any case, I would be very interest to hear about your experience with
> > reduced space option. Have you solved other (smaller) problems before?
> >
> > Good luck,
> >
> > Andreas
> >
> >
> > On Thu, 3 Mar 2005, Andreas Waechter wrote:
> >
> > >
> > >
> > >
> > >
> > >
> > > ----- Forwarded by Andreas Waechter/Watson/IBM on 03/03/2005 07:48 PM -----
> > >
> > > hong weirong
> > >
> > > > cc:
> > > Subject: Re: input executable file,display 'Killed'
> > > 02/27/2005 09:28
> > > PM
> > >
> > >
> > >
> > >
> > >
> > >
> > > Hi, Andreas
> > >
> > > The 'Killed' problem has been solved, because of smaller Jacobian matrix.
> > >
> > > I did not use AMPL and used collocation point to discretize my problem
> > > modelled in Fortran, then used precise differential eguations to get
> > > Jacobin matrix, but no second derivatives. The tatol variables are 280160,
> > > the degrees of freedom are 80, so the reduced space option was selected.
> > > There are 2641464 nonzero elements in Jacobian matrix. Because of small
> > > memory of Linux computer (only 600M), I chose the IRIX64 UNIX computer to
> > > execute my program. The performance of IRIX: 8CPU,57processes, 4096M
> > > Memory. Unfortunately, the following messages were displayed again.
> > >
> > > --------------------------------------------------------------------------------------------------------------------------
> > > estimated double precision work space requirement = 214595842
> > > estimated integer work space requirement = 323441922
> > > Parameter DTOL has multiple assignments - using 0.10000
> > > D-05 .
> > > ******************************************************************************
> > >
> > > This program contains IPOPT, a program for large-scale nonlinear
> > > optimization.
> > > IPOPT is released as open source under the Common Public License (CPL).
> > > For more information visit www.coin-or.org/Ipopt
> > > ******************************************************************************
> > >
> > > ipopt: Error allocating double precision work space: -1
> > > Error initializing algorithmic parameters.
> > > IERR = 96
> > > You might want to check IPOPT.DAT (if you set QCNR>0)
> > > ----------------------------------------------------------------------------------------------------------------------------
> > >
> > > The attachments are config.log and PARAMS.DAT files. I wonder that the
> > > computer has not enough memory? or there are some problems in my approach?
> > > Please give me some suggestions. Thanks a lot.
> > >
> > > best regards,
> > > Hans
> > >
> > >
> > >
> > >
> > >
> > >
> > > Andreas Waechter wrote:
> > > Hans,
> > >
> > > In order for me to help you, you need to tell me more.
> > >
> > > What precisely did you do? Did you use AMPL? How? What is the precise
> > > error message? It might be helpful if you send me the config.log file in
> > > your Ipopt directory (generated when you ran configure).
> > >
> > > Such an error message could happen when you run out of memory. But 20000
> > > variables is not large, unless your matrices (Jacobian and Hessian) are
> > > dense? Are trying to solve a problem modelled in C or Fortran? If so,
> > > what are the number of nonzeros you provide for EV_A and EV_H?
> > >
> > > Your earlier message seemed to indicate that you are not using the GMU
> > > compilers. Is that on purpose? What compiler are you using?
> > >
> > > If you don't care what compiler you are using, you should try to see if
> > > your problem goes away when you use the GNU compilers. For this, please
> > > run
> > >
> > > make distclean
> > > ./configure F77=g77 CC=gcc
> > >
> > > in the Ipopt directory, followed by
> > >
> > > make install
> > >
> > > Andreas
> > >
> > >
> > >
> > > On Wed, 23 Feb 2005, hong weirong wrote:
> > >
> > > > Hi, Andreas
> > > > Thank you for giving me directions. My compilation problem was solved.
> > > > However, I input executable file,display 'Killed' ! I dont know its
> > > > meaning for this message. I add 'FFLAGS=-o -g' and try to use 'GDB'
> > > > debug, the output is
> > > >
> > > > Program terminated with signal SIGKILL, Killed.
> > > > The program no longer exists.
> > > > You can't do that without a process to debug.
> > > >
> > > > Could you give some advices? Maybe too many variables (more than 20000)
> > > ?
> > > >
> > > > regards,
> > > > Hans
> > > >
> > > >
> > > > ---------------------------------
> > > > Do you Yahoo!?
> > > > Yahoo! Mail - 250MB free storage. Do more. Manage less.
> > >
> > >
> > >
> > > Do you Yahoo!?
> > > Yahoo! Mail - now with 250MB free storage. Learn more.(See attached file:
> > > config.log)(See attached file: PARAMS.DAT)
> > >
> > >
> >
> >
> >
> >
> >
> >
> > ---------------------------------
> > Celebrate Yahoo!'s 10th Birthday!
> > Yahoo! Netrospective: 100 Moments of the Web
>
>
>
>
> ---------------------------------
> Do you Yahoo!?
> Yahoo! Small Business - Try our new resources site!
More information about the Coin-ipopt
mailing list