[CHiPPS] Fwd: Questions on Blis

[Alessandro Tiberi]

Ted Ralphs ha scritto:
> We have been testing BLIS with some fairly large instances recently,
> and have also been running into memory problems on long runs with
> small numbers of processors. How many rows do your instances have? How
> many nonzeros?
These are the average stats of our problematic instances:
- 500K rows
- 1M columns
- 2.5M non zero elements

We are running Blis (mpi built) on a single machine with 16Gb Ram and 
two quadcore (opteron), so that's a total
of eight cores. We tried several  degrees of parallelism (# of process), 
from 2 to 8. The bottom line seems to be
that the more processes we use, the sooner it runs out of memory. 
However, from Alp's log, it appears that  relatively few nodes are
queued (about 13K nodes with 2 processes running and 7K with 6.)
(btw, I have been told that famous commercial software has no problem 
with these instances even if running on much lower profile platforms 
although I did not personally try
it)

>  What is the platform you are running it on? We are also
> looking at ways to reduce the memory footprint. Of course,
> distributing the computation over more processors works to some
> degree.
Concerning this, I guess that when using more  processes we run out of 
memory earlier just because there is a certain unavoidable
amount of data duplicated, right?
>  I never really thought about the idea of using standard
> compression software to compress the node description. I would be
> surprised if it had much effect in most cases, since the data are
> already compressed by the use of sparse vectors and matrices
> everywhere and also by storing subtrees by differencing, etc.
I do not have a very strong feeling about this (On the one hand even if 
vectors are sparse they eventually are just a sequence of numbers, and 
hopefully they are far enough from being uniformly distributed, so we 
should gain something from compression...).
However I am not ever sure that node description is the right thing to 
compress... Any idea is greatly appreciated!
Do you happen to know or have an estimation, on average,  of how much 
space it takes to store a node desc, with respect to the original problem?

>  I guess
> it should be easy to try, though. 
I hope it is, I think I will give it a try.
 
> Are you using BLIS out of the box to
> solve instances in MPS format or do you have application-specific
> methods that you are using with BLIS?
>   
Essentially (apart from the addition of some heuristics) out of the box, 
at least for now.

best,

Alessandro