Integration -- was: Re: [Coin-standards] minutes / strategy

[Irv Lustig]

Leo:

At 11:43 AM 4/11/02 -0500, Leonardo B. Lopes wrote:

>Two points here: 1st, the distinction between "we provide solutions for
>that", and "our customers do it all the time", which IMHO is a critical
>point. We've got to be able to do more of the former, and rely less on the
>latter, especially when what the customer is doing is technical analysis.
>And I don't think we can do it unless we update the way programs talk
>optimization to each other.

I don't think the programs need to talk *optimization* to each other.  They 
need to have better ways of passing data to each other.  The optimization 
tool (like a modeling language) can take this data and create the 
optimization instance.


>Point taken. Most of the time, it is easier to provide linkage to models
>than to data. But there are cases in which you have to manipulate the
>instance _despite_ having access to a modeling language or library. I'm
>working on a case like that right now. This situation arises more and
>more as you try to provide the user more abstraction. More on this
>later...

In my experience, if you have a modeling language representation of a 
problem, you manipulate an instance when designing new algorithms to solve 
that instance.  A standard helps here if you're an algorithm designer.  But 
the end user was using a modeling language, so the standard is something 
that is "under the hood" when using the modeling language.


>The standard can be easier to manipulate if we design it with that goal in
>mind. MPS was designed to be read using card readers or tape. Simply
>assuming random access would make a tremendous difference.

For whom?

>  It allows
>people to manage ultra-large instances, do decompositions, convey problem
>structure, etc...

All the benefits above seem to benefit algorithm designers.  Maybe this is 
where the disconnect is happening?

>Having it be xml based also helps with network
>communication,

Send your gigabyte instance via the network?  No way.  :->

>  with data extraction from other programs, with data
>exporting into other programs, etc...

I don't see how a standard for representing optimization instances connects 
with data importing and exporting from other programs.


>I don't expect to be creating instances bypassing a modeling language or
>library very often, no matter what we have for a standard. But I do expect
>to be collaborating with a modeling language or library in order to better
>insulate the modeler of some of the things going on. Modeling languages
>can't and shouldn't do everything. More on that later...

It seems to be a separate topic.

>Making it easier to integrate external data into optimization models is
>sufficient motivation for me. I think making it easier to integrate
>external data implies that it is easier to integrate optimization into
>non-optimization applications.

I have to admit that I am missing the connection between "integrating 
external data" and "having a standard for representing optimization 
instances".  I do agree that if it is easier to integrate external data, 
then it is easier to integrate optimization into non-optimization 
applications.  I just don't see how the standard for representing 
optimization problems helps solve the external data problem.

>  I also think that the more people do that,
>the more people will be using OPL-Studio, Xpress-Mosel and Optimax. The
>people who develop and sell those tools are those who will most benefit
>from what we are trying to do.

I'm not sure I agree with you here that as the tool vendor, we benefit by 
having a new standard.

>  If academia benefits the most, that is fine
>with me too. I think everybody benefits.
>
>By the way, I haven't had the opportunity to thank you (Irv) for your
>insights. I've been learning a lot and changing a bit my assumptions about
>some of these issues. I hope you're not done, although I'm anxious to get
>technical soon. I'll try to do that next...

It's an interesting discussion.  I hope I'm not dampening your enthusiasm 
by giving you my thoughts from the trenches.

         -Irv