[Cbc] Barrier method SEGV (out of memory) -- increase memory limit?

[John Forrest]

Martin,

Looking at the last line - flop count 2.20667e+16, the code would take 
something of the order of 12 days per iteration (maybe several times 
that with cache misses) - you might get away with less than 6 months 
total.  I would expect Gurobi to have as good a sparse ordering so 700G 
does make it impossible.

There probably is some structure that can be used.  You could also try
...... -idiot 100 -primals
and see what that looks like.

If you send me a small version, I could have a quick look - or if there 
is somewhere you can put compressed model (before Tuesday) I could 
download it - and solve it on my laptop :-)

John Forrest
On 05/04/14 15:32, Martin Koutecký wrote:
> Hello,
>
> I'm trying to solve a big LP (continuous) problem using the barrier 
> method, but I'm getting "Segmentation fault" some time after presolve 
> (before optimization starts, seems like).
>
> When I look at the memory usage in htop it goes through 3 stages:
>
> 1. VIRT = RES = ~8GB after the problem is loaded, during presolve
> 2. VIRT = 13G, RES = 8-10G, after this the line "1.70542e+09 elements 
> in sparse Cholesky, flop count 2.20667e+16" is printed.
> 3. VIRT = 20G, RES rises from 7G to 20G and then cbc crashes.
>
> Presolve writes this:
> Presolve 1588289 (-6281501) rows, 4931364 (-6482760) columns and 
> 35692658 (-21407980) elements
>
> My questions are these:
> - since I have more memory (128G) then the 20G it allocates (and runs 
> out of), can I increase the limit somehow (manually)?
> - or is the problem too big anyway?
>
> (Gurobi's barrier solver crashes on it with something like "would need 
> ~700G for factorization")
>
> The dual simplex solver has been working on the problem for the past 
> few days (and is slowly making progress). Curiously enough, it's doing 
> better than both Gurobi and CPLEX (dual simplex). (Maybe I don't know 
> how to tune their parameters.) GLPK is completely lost.
>
> Thanks for any help,
>
> Martin
>
> ---------------
>
> The full output is this:
>
>
> Welcome to the CBC MILP Solver
> Version: 2.8.9
> Build Date: Apr  5 2014
>
> command line - [...]cbc/Cbc-2.8.9/Cbc/src/.libs/lt-cbc 
> [...]14989-pulp.mps barr printingOptions rows solution 
> /tmp/14989-pulp_barr.sol (default strategy 1)
> At line 2 NAME          MODEL
> At line 3 ROWS
> At line 7869795 COLUMNS
> At line 64970506 RHS
> At line 72840297 BOUNDS
> At line 84254422 ENDATA
> Problem MODEL has 7869790 rows, 11414124 columns and 57100638 elements
> Coin0008I MODEL read with 0 errors
> Presolve 1588289 (-6281501) rows, 4931364 (-6482760) columns and 
> 35692658 (-21407980) elements
> 1.70542e+09 elements in sparse Cholesky, flop count 2.20667e+16
> Segmentation fault
> _______________________________________________
> Cbc mailing list
> Cbc at list.coin-or.org
> http://list.coin-or.org/mailman/listinfo/cbc
>
>