<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none"><!--P{margin-top:0;margin-bottom:0;} p
{margin-top:0;
margin-bottom:0}--></style>
</head>
<body dir="ltr" style="font-size:12pt;color:#003366;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<p><br>
</p>
<p><br>
</p>
<div id="Signature">
<div name="divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:; margin:0">
<div name="divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; margin:0px">
<span style="background-color:rgb(255,255,255)"><font color="#000080"><b><span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px"><br>
</span></b></font></span></div>
<div name="divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; margin:0px">
<span style="background-color:rgb(255,255,255)"><font color="#000080"><b><span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px"><br>
</span></b></font></span></div>
<div name="divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; margin:0px">
<span style="background-color:rgb(255,255,255)"><font color="#000080"><b><span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px"><br>
</span></b></font></span></div>
<hr style="font-family:Calibri,Arial,Helvetica,sans-serif">
<div name="divtagdefaultwrapper" style="margin:0px"><span style="font-family:Calibri,Arial,Helvetica,sans-serif; background-color:rgb(255,255,255)"><font color="#000080"><b><span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px">Ana
María Barragán Montero</span><span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px">, </span></b></font></span>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif"><span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)"><font color="#333399">PhD student</font></span><br style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)">
<font color="#333399"><span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)">Université catholique de Louvain</span><br style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)">
<span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)">Molecular Imaging, Radiotherapy and Oncology (MIRO)</span></font></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif"><font color="#333399"><span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)">Avenue Hippocrate 54 </span><br style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)">
<span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)">B-1200 Bruxelles, Belgium</span><br style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)">
<span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)">ana.barragan@uclouvain.be</span></font></div>
<div><span style="background-color:rgb(255,255,255)"><font face="Segoe UI WPC, Segoe UI, Tahoma, Microsoft Sans Serif, Verdana, sans-serif" color="#333399" size="2">+32(0) 27649527</font></span></div>
</div>
</div>
</div>
<div style="font-size:12pt; color:#003366; background-color:#FFFFFF; font-family:Calibri,Arial,Helvetica,sans-serif" dir="ltr">
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> Ana Maria Barragan Montero<br>
<b>Sent:</b> Friday, August 14, 2015 4:45 PM<br>
<b>To:</b> Ipopt User<br>
<b>Subject:</b> Re: [Ipopt] Objective function decreases extremely slowly</font>
<div> </div>
</div>
<div>
<p>Hi!</p>
<p><br>
</p>
<p>Thanks for your quick answer. </p>
<p><br>
</p>
<p>I did run the derivative checker at the very beginning to check that my implementation was right and everything seems to be OK. Moreover, I guess that if the derivatives are not well calculated I would not get right results, isn't it? The results I get are
OK, the problem is that I need a lot of iterations to achieve it. </p>
<p><br>
</p>
<p>About my objective and constraint functions, ...here we go:</p>
<p>I work in the field of radiation therapy for cancer patients. In my problem I need to find the optimal dose for the patient which is obtained by a weighted sum of small proton beams.
</p>
<ul>
<li>The weights for all these beamlets are my <strong>variables of optimization</strong>:
<em><strong>w = [</strong>w</em><sub><em>1</em></sub><em> w</em><sub><em>2</em></sub><em> w</em><sub><em>3
</em></sub><em>...w</em><sub><em>8500</em></sub><em><strong>]. </strong></em>(with values usually between 0 and 10).
</li><li>I need to be robust to different scenarios s (due to patient movement), so the dose to each scenario is then: d(s) =
<span style="font-family:Symbol,"Segoe UI",Tahoma,sans-serif"></span>w*P(s), where P is a matrix mapping the weights to dose in scenario s.
<br>
</li><li>I want to have a minimum dose in the target and no more than a certain dose in the organs, so I create a function to be minimized which is a weighted sum of dose objectives and will be my
<strong>constraint function:</strong><br>
</li></ul>
<strong>f (d(w,s)) = impw</strong><sub><strong>1</strong></sub><strong>*f</strong><sub><strong>1</strong></sub><strong> +impw</strong><sub><strong>2</strong></sub><strong>*f</strong><sub><strong>2</strong></sub><strong>+...
</strong><br>
<br>
where <em>impw</em><sub><em>i</em></sub> is an 'important weight' which increases according to the importance that I give to the objective
<em>i, </em><br>
and f<sub>i </sub>are the dose objective functions which depend on the dose at each scenario, thus f<sub>i</sub>(d(w,s))<br>
<br>
<ul>
<li>I attempt to be robust to all scenarios by being robust to the worst of them and I introduce a auxiliary variable t, so the problem to solve is the following:
</li></ul>
<p></p>
<p><br>
</p>
<p> minimize t<br>
</p>
<p> <em><strong>w,</strong></em><strong> </strong><em>t</em></p>
<p><br>
</p>
<p> subject to t => f(d(w,s)) for all scenarios s</p>
<p> w => 0</p>
<p><br>
</p>
<p> And I will have as many constraint functions as the number of scenarios we have which is usually <=20.</p>
<p><br>
</p>
<p><br>
</p>
<p>IMPORTANT note: I realised that if I change the impw variables to a small values (<10) the objective function is changing more dinamically and faster than before, where I was using bigger values (500-1000). But anyway I need more than 500-1000 iterations
to get a good result. Do you have any idea of what can I do to reduce the number of iterations?</p>
<p><br>
</p>
<p><br>
</p>
<p>Thank you in advance, </p>
<p><br>
</p>
<p>Ana<br>
</p>
<p><br>
</p>
<p><br>
<em><strong></strong></em></p>
<p><br>
</p>
<p><br>
</p>
<p><br>
</p>
<div id="Signature">
<div name="divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:; margin:0">
<div name="divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; margin:0px">
<span style="background-color:rgb(255,255,255)"><font color="#000080"><b><span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px"><br>
</span></b></font></span></div>
<div name="divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; margin:0px">
<span style="background-color:rgb(255,255,255)"><font color="#000080"><b><span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px"><br>
</span></b></font></span></div>
<div name="divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; margin:0px">
<span style="background-color:rgb(255,255,255)"><font color="#000080"><b><span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px"><br>
</span></b></font></span></div>
<hr style="font-family:Calibri,Arial,Helvetica,sans-serif">
<div name="divtagdefaultwrapper" style="margin:0px"><span style="font-family:Calibri,Arial,Helvetica,sans-serif; background-color:rgb(255,255,255)"><font color="#000080"><b><span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px">Ana
María Barragán Montero</span><span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px">, </span></b></font></span>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif"><span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)"><font color="#333399">PhD student</font></span><br style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)">
<font color="#333399"><span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)">Université catholique de Louvain</span><br style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)">
<span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)">Molecular Imaging, Radiotherapy and Oncology (MIRO)</span></font></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif"><font color="#333399"><span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)">Avenue Hippocrate 54 </span><br style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)">
<span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)">B-1200 Bruxelles, Belgium</span><br style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)">
<span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)">ana.barragan@uclouvain.be</span></font></div>
<div><span style="background-color:rgb(255,255,255)"><font face="Segoe UI WPC, Segoe UI, Tahoma, Microsoft Sans Serif, Verdana, sans-serif" color="#333399" size="2">+32(0) 27649527</font></span></div>
</div>
</div>
</div>
<div style="color:rgb(33,33,33)">
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> Ipopt User <ipoptuser@gmail.com><br>
<b>Sent:</b> Friday, August 14, 2015 3:31 PM<br>
<b>To:</b> Ana Maria Barragan Montero<br>
<b>Subject:</b> Re: [Ipopt] Objective function decreases extremely slowly</font>
<div> </div>
</div>
<div>
<div dir="ltr">Have you run the derivative checker? What do the objective and constraint functions look like?</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Fri, Aug 14, 2015 at 5:25 AM, Ana Maria Barragan Montero
<span dir="ltr"><<a href="mailto:ana.barragan@uclouvain.be" target="_blank">ana.barragan@uclouvain.be</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex; border-left:1px #ccc solid; padding-left:1ex">
<div dir="ltr" style="font-size:12pt; color:#003366; background-color:#ffffff; font-family:Calibri,Arial,Helvetica,sans-serif">
<p><br>
</p>
<p>Dear all, </p>
<p><br>
</p>
<p>I am using IPOPT (through Matlab interface) with the option <em>limited memory bfgs hessian approximation</em> to solve a problem of the following characteristics (the total number of variables can change but it is often between 5000 and 10000).
<br>
</p>
<p><br>
</p>
<p>*************************************************************************<br>
This is Ipopt version 3.11.8, running with linear solver ma57.<br>
<br>
Number of nonzeros in equality constraint Jacobian...: 0<br>
Number of nonzeros in inequality constraint Jacobian.: 60613<br>
Number of nonzeros in Lagrangian Hessian.............: 0<br>
<br>
Total number of variables............................: 8659<br>
variables with only lower bounds: 8659<br>
variables with lower and upper bounds: 0<br>
variables with only upper bounds: 0<br>
Total number of equality constraints.................: 0<br>
Total number of inequality constraints...............: 7<br>
inequality constraints with only lower bounds: 7<br>
inequality constraints with lower and upper bounds: 0<br>
inequality constraints with only upper bounds: 0</p>
<p><br>
</p>
<p>**************************************************************************<br>
</p>
<p><br>
</p>
<p>Here below you can find the ipopt options I am using (those which do not appear are set by the program to their default value)</p>
<p><br>
</p>
<p> % Initialization<br>
options.ipopt.bound_frac = 0.01; <br>
options.ipopt.bound_push = 0.001;</p>
<p><br>
</p>
<p>options.ipopt.dual_inf_tol = 1; <br>
options.ipopt.constr_viol_tol = 0.0001; <br>
options.ipopt.compl_inf_tol = 0.0001; <br>
options.ipopt.acceptable_constr_viol_tol = 0.01; <br>
<br>
% NLP scaling<br>
options.ipopt.nlp_scaling_max_gradient = 100; <br>
</p>
<p><br>
</p>
<p>% Quasi-Newton<br>
options.ipopt.hessian_approximation = 'limited-memory'; <br>
options.ipopt.limited_memory_update_type = 'bfgs';<br>
<br>
% Barrier parameter<br>
options.ipopt.mu_strategy = 'monotone';<br>
options.ipopt.mu_init = 0.1; <br>
</p>
<p><br>
</p>
<p><br>
</p>
<p>My problem is that I need a lot of iterations (more than 1500) to get a good result because the objective function (which has a very big value in the beginning) is decreasing extremely slowly. As an example here you have the print output for the iteration
10 and 500:</p>
<p><br>
</p>
<p>iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_pr ls<br>
</p>
<p> 10 5.4760000e+10 0.00e+00 6.20e+02 -0.2 3.33e+02 - 6.13e-02 1.23e-02f 1 sigma=1.00e-06 qf=13Ws</p>
<p><br>
</p>
<p> 500 2.9078333e+10 1.38e+10 1.00e+00 -2.9 9.18e+13 - 4.66e-09 3.06e-10f 1 sigma=6.04e-01 qf=12Wp</p>
<p><br>
</p>
<p><br>
</p>
<p>I wonder if I could use any other ipopt options to make the objective function decrease faster so that I can reduce the number of iterations. Moreover, I am using ma57 solver, do you recommend me any other (as PARDISO for example) for large problems as this
one? </p>
<p>I guess that it is normal that the program takes longer at the beginning to find the good search direction since it does not have the information about the hessian because I am using the bfgs approx. In my case I have the information to calculate analytically
the hessian. To calculate it at each iteration would increase considerably the computation time, however, I could calculate it at the beginning and thus initialize the hessian to the right value. Do you think that would reduce the number of iterations?
<br>
</p>
<p><br>
</p>
<p>I am new with IPOPT and I don't have much experience in the field of optimization so please let me know if you need more information about my problem (formulation, variables, etc..) that can help you to find a solution.</p>
<p><br>
</p>
<p>Thank you in advance, </p>
<p><br>
</p>
<p>Ana<br>
</p>
<div>
<div name="divtagdefaultwrapper">
<div name="divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; margin:0px">
<span style="background-color:rgb(255,255,255)"><font color="#000080"><b><span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px"><br>
</span></b></font></span></div>
<div name="divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; margin:0px">
<span style="background-color:rgb(255,255,255)"><font color="#000080"><b><span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px"><br>
</span></b></font></span></div>
<hr style="font-family:Calibri,Arial,Helvetica,sans-serif">
<div name="divtagdefaultwrapper" style="margin:0px"><span style="font-family:Calibri,Arial,Helvetica,sans-serif; background-color:rgb(255,255,255)"><font color="#000080"><b><span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px">Ana
María Barragán Montero</span><span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px">, </span></b></font></span>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif"><span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)"><font color="#333399">PhD student</font></span><br style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)">
<font color="#333399"><span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)">Université catholique de Louvain</span><br style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)">
<span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)">Molecular Imaging, Radiotherapy and Oncology (MIRO)</span></font></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif"><font color="#333399"><span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)">Avenue Hippocrate 54 </span><br style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)">
<span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)">B-1200 Bruxelles, Belgium</span><br style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)">
<span style="font-family:'Segoe UI WPC','Segoe UI',Tahoma,'Microsoft Sans Serif',Verdana,sans-serif; font-size:13px; background-color:rgb(255,255,255)"><a href="mailto:ana.barragan@uclouvain.be" target="_blank">ana.barragan@uclouvain.be</a></span></font></div>
<div><span style="background-color:rgb(255,255,255)"><font face="Segoe UI WPC, Segoe UI, Tahoma, Microsoft Sans Serif, Verdana, sans-serif" color="#333399" size="2"><a href="tel:%2B32%280%29%2027649527" value="+3227649527" target="_blank">+32(0) 27649527</a></font></span></div>
</div>
</div>
</div>
</div>
<br>
_______________________________________________<br>
Ipopt mailing list<br>
<a href="mailto:Ipopt@list.coin-or.org">Ipopt@list.coin-or.org</a><br>
<a href="http://list.coin-or.org/mailman/listinfo/ipopt" rel="noreferrer" target="_blank">http://list.coin-or.org/mailman/listinfo/ipopt</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</div>
</body>
</html>