<div dir="ltr">Contrary to popular opinion, file-based I/O is usually not a bottleneck when interfacing with solvers. <a href="https://en.wikipedia.org/wiki/Nl_(format)" target="_blank">NL format</a> used by AMPL and Pyomo is extremely compact and efficient. For example, a simple problem with 1000 variables and 1000 constraints and a 1000x1000 dense constraint matrix<div><br></div><div><div> var x{1..1000};</div><div> s.t. c{j in 1..1000}: sum{i in 1..1000} Uniform01() * x[i] = j;</div></div><div><br></div><div>takes about 11 MiB which is little more than the storage required for the constraint matrix alone.</div><div><br></div><div>Best regards,</div><div>Victor</div><div dir="ltr"><div><div><br><div class="gmail_quote"><div dir="ltr">On Wed, Jul 8, 2015 at 9:38 AM Meketon, Marc <<a href="mailto:Marc.Meketon@oliverwyman.com" target="_blank">Marc.Meketon@oliverwyman.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I’ve just begun to look into Pyomo, and I asked their user-group if it directly connect to CBC (and I presume CLP) (<a href="https://groups.google.com/forum/#!topic/pyomo-forum/60G-vDRSLbA" target="_blank">https://groups.google.com/forum/#!topic/pyomo-forum/60G-vDRSLbA</a>).
The answer from Gabriel Hackebeil is that Pyomo uses file-based I/O, so I imagine it does not solve support the “hundreds of gigabytes” issue that Matt has.
<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> <a href="mailto:clp-bounces@list.coin-or.org" target="_blank">clp-bounces@list.coin-or.org</a> [mailto:<a href="mailto:clp-bounces@list.coin-or.org" target="_blank">clp-bounces@list.coin-or.org</a>]
<b>On Behalf Of </b>Matthew Gidden<br>
<b>Sent:</b> Wednesday, July 08, 2015 12:00 PM<br>
<b>To:</b> Matt Bromberg<br>
<b>Cc:</b> <a href="mailto:clp@list.coin-or.org" target="_blank">clp@list.coin-or.org</a><br>
<b>Subject:</b> Re: [Clp] Dense LP Problem Interface<u></u><u></u></span></p></div></div><div lang="EN-US" link="blue" vlink="purple"><div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">Hi Matt,<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Have you checked out <a href="https://software.sandia.gov/trac/pyomo" target="_blank">
Pyomo</a>? It's a python mathematical programming interface from Sandia that can call a number of solvers (namely, COIN CBC for this case). I admit that I haven't used it much myself (yet), but it looks promising and seems to have some overlap with the COIN-OR
community as well.<u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Wed, Jul 8, 2015 at 10:53 AM, Matt Bromberg <<a href="mailto:mattcbro@earthlink.net" target="_blank">mattcbro@earthlink.net</a>> wrote:<u></u><u></u></p>
<p class="MsoNormal">I have yet to see a simple way to use CBC or CLP using raw binary data. In particular I have a large problem with 40K constraints 10K variables or so.<br>
That's about 3Gbytes of raw doubles. It expands to an unwieldy size using any of the text base input formats. (hundreds of gigabytes).<br>
<br>
I can get that data into a python numpy array but it does not appear to be easy to dump that raw array into CLP. Unfortunately cylp doesn't support the latest API.<br>
I would roll my own python interface but there is literally no documentation I can find online for the C interface to something like CoinMP.<br>
<br>
All the other interfaces appear to be useless for my purposes. Pulp is rather arcane and there is no obvious way to pull the data back into python.<br>
<br>
I am in somewhat disbelief that I can't do this:<br>
[xopt,fmin] = linprog(c, Acon, rhsvec) ;<br>
<br>
to solve min c' * x given Acon * x <= rhsvec .<br>
<br>
The above is the one line matlab interface to linprog. There should be something similar in Python in support of CLP using it's primary matrix array interface, numpy/ndarrays.<br>
Even my cylp solution takes 10 lines of python code to do this, plus half a day of parsing documentation. It's just unfortunate that it segfaults on the larger problems.<br>
<br>
If anyone has a good way to do this in Python or C, with an example I'd love to see it. For now I have to move on to other solvers.<br>
_______________________________________________<br>
Clp mailing list<br>
<a href="mailto:Clp@list.coin-or.org" target="_blank">Clp@list.coin-or.org</a><br>
<a href="http://list.coin-or.org/mailman/listinfo/clp" target="_blank">http://list.coin-or.org/mailman/listinfo/clp</a><u></u><u></u></p>
</div>
<p class="MsoNormal"><br>
<br clear="all">
<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal">-- <u></u><u></u></p>
<div>
<div>
<div>
<div>
<p class="MsoNormal">Matthew Gidden, Ph.D.<br>
Postdoctoral Associate, Nuclear Engineering<u></u><u></u></p>
<div>
<p class="MsoNormal">The University of Wisconsin -- Madison<br>
Ph. 225.892.3192<u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div></div><div lang="EN-US" link="blue" vlink="purple"><div></div>
<br>
<hr>
<font face="Arial" color="Gray" size="1">This e-mail and any attachments may be confidential or legally privileged. If you received this message in error or are not the intended recipient, you should destroy the e-mail message and any attachments or copies,
and you are prohibited from retaining, distributing, disclosing or using any information contained herein. Please inform us of the erroneous delivery by return e-mail. Thank you for your cooperation.<br>
</font>
</div>
_______________________________________________<br>
Clp mailing list<br>
<a href="mailto:Clp@list.coin-or.org" target="_blank">Clp@list.coin-or.org</a><br>
<a href="http://list.coin-or.org/mailman/listinfo/clp" rel="noreferrer" target="_blank">http://list.coin-or.org/mailman/listinfo/clp</a><br>
</blockquote></div></div></div></div></div>