[Ipopt] IPOPT runs out of memory for analytical jacobian of theconstraints

[Tony Kelman]

The replies about copy-on-write and Matlab tricks for memory management are useful if you’re stuck using that environment (though I encourage you to consider looking into modern, open-source, high-performance alternatives that are available these days), but likely overthinking the problem. A 32k by 32k dense matrix of doubles would take over 8 gigabytes. The same size sparse matrix would take 12 bytes per nonzero element, plus 128 kb for the column pointers (or 16 bytes per nonzero + 256 kb if the indices are saved as 64 bit ints). So most likely, somewhere in your code is either initially allocating or accidentally converting your Jacobian matrix to dense. I haven’t used Matlab’s numjac, but I’m going to guess that if it worked at all for a 32k-variable, 32k-constraint problem then it’s outputting sparse results.

-Tony


From: Hashemi, Reza 
Sent: Thursday, December 18, 2014 6:49 AM
To: ipopt at list.coin-or.org 
Subject: [Ipopt] IPOPT runs out of memory for analytical jacobian of theconstraints

Hello everyone,

 

I am facing a problem in using IPOPT which some of you might have experience with it.

In the context of the optimal control, I am using the IPOPT to solve a huge NLP problem with 32,005 variables. Since the model is very complex, up to now, I have used the numeric Jacobians. I provide the numeric Jacobians using the MATLAB internal function numjac and I am able to produce some results.

Since I need to accelerate the convergence of the algorithm, I have created a function which supplies the analytical Jacobian of the constraints (The Jacobian has a size of 32000x32005).

Surprisingly, when I am using the analytical Jacobian, the IPOPT runs out of memory or becomes torturingly slow.

I am pretty sure that the analytical Jacobian is correct. Moreover, I compare the analytical Jacobian and the numeric one at each step and do not see a large difference…

 

Does anyone know what might be the possible reason for this strange behavior of IPOPT?

 

Thanks in advance.

Reza

 

 

Reza Hashemi, M.Sc.

Lehrstuhl für Systemdynamik und Prozessführung / Process Dynamics and Operations Group

Fakultät Bio- und Chemieingenieurwesen / Biochemical and Chemical Engineering Department

TU Dortmund

Emil-Figge-Str. 70

44221 Dortmund, Germany

 

Phone: (+49/0) 231 755 5165                                          Fax:       (+49/0) 231 755 5129

www.dyn.bci.tu-dortmund.de

 

 



--------------------------------------------------------------------------------
_______________________________________________
Ipopt mailing list
Ipopt at list.coin-or.org
http://list.coin-or.org/mailman/listinfo/ipopt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://list.coin-or.org/pipermail/ipopt/attachments/20141219/eb136906/attachment-0001.html>