[Clp] coinpackedmatrix appendCol taking a very long time

[John Forrest]

David,

appendCol may be doing a delete/new at each call - this may cause locks.

Assuming you have a reasonable upper limit on size of matrix, I would 
allocate enough memory to build matrix and use addColumns to do all at 
once.  Even if you have not got much idea of size then you could add in 
chunks.

John Forrest
On 26/09/16 12:27, David Prime wrote:
> Hi All,
>
> I've been using CLP for a little while and am finding it to be very 
> good, proving fast and stable results.
>
> One issue I've seen is that when I'm creating and executing many 
> models in parallel (we build and execute perhaps a few hundred models 
> per second across 16 cores in one program) the ratio of time spent 
> creating the model and then executing it goes from 1:1 to 10:1 under 
> heavy load when we attempt to run the same code on a 32 core machine 
> with work spread evenly across all cores. Execution times remain 
> pretty much static, only increasing with model complexity or when the 
> machine running it is maxing out all available cores and the program 
> becomes CPU starved.
>
> The model creation time is spent almost exclusively in the
> CoinPackedMatrix->appendCol(
> const int vecsize,
> const int * vecind,
> const double * vecelem)
>
> method. I'm wondering if there's any internal global locks in any of 
> the libraries involved here that might be making performance worse as 
> parallelism increases. Another theory I have is that we only want to 
> increase parallelism from 16 to 32 cores when we're under very heavy 
> load so the characteristics of the models may also be changing in 
> terms of the size of the matrix we want to construct. Is there a more 
> efficient way to allocate the matrix than using appendCol?
>
> Any thoughts?
>
> Thanks,
> David Prime
>
>
> _______________________________________________
> Clp mailing list
> Clp at list.coin-or.org
> https://urldefense.proofpoint.com/v2/url?u=http-3A__list.coin-2Dor.org_mailman_listinfo_clp&d=CwICAg&c=Ngd-ta5yRYsqeUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&r=js2M0T-3OIMIVDvokcKjokJbk0F8QOCd0mT4FsVFE88&m=pHOYvsXa0_Zqv_U15oE55jSNMep6TTkDVkrMY4xiDPo&s=5axPjeE3O9TC6t0mmqjIipCHDnEdWSRmLXC1aRvc9Tg&e=


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://list.coin-or.org/pipermail/clp/attachments/20160926/13df2a17/attachment.html>