[Ipopt] Fwd: IPOPT runs out of memory for analytical jacobian of the constraints

[Jason Moore]

And this.

Jason
moorepants.info
+01 530-601-9791

---------- Forwarded message ----------
From: Steven Dirkse <sdirkse at gams.com>
Date: Thu, Dec 18, 2014 at 12:52 PM
Subject: Re: [Ipopt] IPOPT runs out of memory for analytical jacobian of
the constraints
To: Jason Moore <moorepants at gmail.com>

Jason,

I believe Matlab uses copy-on-write when copies of matrices are made, and
that this includes copies made as a result of a function call.  They treat
structures somewhat differently (each element of the structure is
independent when it comes to copying) and if you use GLOBAL variables you
don't get copy semantics anyway.  So if you are not modifying inputs to
functions no expensive copy is made.

Here's a relevant, if old, link:

http://blogs.mathworks.com/loren/2006/05/10/memory-management-for-functions-and-variables/

-Steve

On Thu, Dec 18, 2014 at 10:53 AM, Jason Moore <moorepants at gmail.com> wrote:
>
> Matlab often does dumb things and copies arrays. For example, if you pass
> a matrix to a function it is copied. This leads to large memory use when
> you work with bigger data. You should carefully examine your memory use in
> the Matlab code you are using and see where you are using all the memory.
> It is likely that you can use some Matlab "tricks" to prevent copying.
>
>
> Jason
> moorepants.info
> +01 530-601-9791
>
> On Thu, Dec 18, 2014 at 9:49 AM, Hashemi, Reza <
> Reza.Hashemi at bci.tu-dortmund.de> wrote:
>
>> Hello everyone,
>>
>>
>>
>> I am facing a problem in using IPOPT which some of you might have
>> experience with it.
>>
>> In the context of the optimal control, I am using the IPOPT to solve a
>> huge NLP problem with 32,005 variables. Since the model is very complex, up
>> to now, I have used the numeric Jacobians. I provide the numeric Jacobians
>> using the MATLAB internal function *numjac* and I am able to produce
>> some results.
>>
>> Since I need to accelerate the convergence of the algorithm, I have
>> created a function which supplies the analytical Jacobian of the
>> constraints (The Jacobian has a size of 32000x32005).
>>
>> Surprisingly, when I am using the analytical Jacobian, the IPOPT runs out
>> of memory or becomes torturingly slow.
>>
>> I am pretty sure that the analytical Jacobian is correct. Moreover, I
>> compare the analytical Jacobian and the numeric one at each step and do not
>> see a large difference…
>>
>>
>>
>> Does anyone know what might be the possible reason for this strange
>> behavior of IPOPT?
>>
>>
>>
>> Thanks in advance.
>>
>> Reza
>>
>>
>>
>>
>>
>> *Reza Hashemi, M.Sc.*
>>
>> Lehrstuhl für Systemdynamik und Prozessführung / Process Dynamics and
>> Operations Group
>>
>> Fakultät Bio- und Chemieingenieurwesen / Biochemical and Chemical
>> Engineering Department
>>
>> TU Dortmund
>>
>> Emil-Figge-Str. 70
>>
>> 44221 Dortmund, Germany
>>
>>
>>
>> Phone: (+49/0) 231 755 5165
>> Fax:       (+49/0) 231 755 5129
>>
>> www.dyn.bci.tu-dortmund.de
>>
>>
>>
>>
>>
>> _______________________________________________
>> Ipopt mailing list
>> Ipopt at list.coin-or.org
>> http://list.coin-or.org/mailman/listinfo/ipopt
>>
>>
> _______________________________________________
> Ipopt mailing list
> Ipopt at list.coin-or.org
> http://list.coin-or.org/mailman/listinfo/ipopt
>
>

-- 
Steven Dirkse, Ph.D.
GAMS Development Corp.,  Washington DC
Voice: (202)342-0180     Fax: (202)342-0181
sdirkse at gams.com
http://www.gams.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://list.coin-or.org/pipermail/ipopt/attachments/20141218/b236ea56/attachment.html>