[Coin-discuss] New low level IO for CoinMpsIO

[John J Forrest]

Benjamin,

Thanks very much for your contribution.  It makes the code cleaner and 
gives me a speed up of about five.  I will test it for memory leaks etc 
and give it some more tests and then put it into cvs.  I will fix the 
minor TO-DOS as well.

I will be contacting you about the necessary paperwork!

John Forrest



Benjamin Hummel <hummelb at in.tum.de> 
Sent by: coin-discuss-bounces at list.coin-or.org
04/10/2005 04:38 PM
Please respond to
Discussions about open source software for Operations Research 


To
Discussions about open source software for Operations Research 
<coin-discuss at list.coin-or.org>
cc

Subject
[Coin-discuss] New low level IO for CoinMpsIO






Hi,

using CoinMpsIO for reading gzip'ed MPS files seemed pretty slow to me:

To illustrate this, some times collected with the mpsread program 
(attached, but without Makefile) on my machine.

fast0507 (MIPLIB): 1.25 sec uncompressed, 6.12 sec gzip'ed
big internal MPS:  3.25 sec uncompressed, 14.46 sec gzip'ed

The reason for this is that MpsIO uses the gzgets function for gzip 
input, which is _extremely_ inefficient (as it reads every single byte 
by a separate call to gzread). The solution is pretty easy: implement a 
better function for linewise reading from gzip files (also based on 
gzread, but using another buffer). This however is not easy to do with 
the current MpsIO implementation, as file IO is scattered all over the 
class.

A possible solution would be to isolate the low level IO stuff from the 
MPS handling. The MpsIO is then based on a file IO class (CoinFileInput 
resp. CoinFileOutput). I implemented this and attached the modified 
code. The only files affected are CoinMpsIO.*, and new CoinFileIO.* are 
added. The code is based on the current CVS version of COIN. To compile 
successfully the CoinFileIO.cpp has to be added to Coin/Makefile. As 
this is now very simple, bzip2 support was added, too.

To give you an idea of the performance of the new code:

fast0507 (MIPLIB): 1.19 sec uncompressed, 1.41 sec gzip'ed
big internal MPS:  3.16 sec uncompressed, 3.72 sec gzip'ed

Note that this code is not yet fully tested (although the unitTest still 
works; are there any other/better tests?) and reading from stdin is not 
supported (although easy to implement).

It would be nice, to see this code included into the "official" version 
of Coin (maybe after some modifications, like adding stdin support). 
Just contact me for any actions required.

Regards,
Benjamin


#include <iostream>

#include <CoinMpsIO.hpp>

using namespace std;

int main (int argc, char **argv)
{
  if (argc <= 1) {
    cerr << "No args given" << endl;
    return 1;
  }

  cout << "Loading " << argv[1] << endl;
  CoinMpsIO mpsIO;
  int num_errors = mpsIO.readMps (argv[1], "");

  if (num_errors != 0)
    cout << "Had " << num_errors << " errors" << endl;
  else
    cout << "Done (" << mpsIO.getNumRows() << " rows, " 
                  << mpsIO.getNumCols () << " cols)" << endl;

  return 0;
}

_______________________________________________
Coin-discuss mailing list
Coin-discuss at list.coin-or.org
http://list.coin-or.org/mailman/listinfo/coin-discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://list.coin-or.org/pipermail/coin-discuss/attachments/20050411/2a460bae/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: new_mpsio.tar.gz
Type: application/octet-stream
Size: 39906 bytes
Desc: not available
URL: <http://list.coin-or.org/pipermail/coin-discuss/attachments/20050411/2a460bae/attachment.obj>