[Coin-standards] minutes / strategy

[Leonardo B. Lopes]

Great! At least one person read that long message :)... It'd be nice to
get other people's opinions on some of these things... I'll try to be more
terse this time...

On Tue, 9 Apr 2002, Irv Lustig wrote:
> standard. However, I work for a vendor, and the users of optimization 
> software aren't banging down our doors asking for a new standard.  So, from 
> where I sit, there is not a "standards problem" for those users of 
> optimization.
> 

I didn't necessarily expect them to do that very often, especially to you.
I probably wouldn't complain about that issue to you if I were an user
either. But I wonder if they complain about some related things: Do you
hear complaints about users who wished they could do some Oracle
integration? Or who could really benefit from running SPSS or Mathematica
over some data before using it in an optimization code? Or who would like
to develop web interfaces to their applications? Do you think you might be
able to sell more cplex licenses if people didn't have to install it on
their own machines?

I think that _part_ of creating more satisfactory answers to those
questions, and thus creating new growth opportunities and new research
material for optimization, is coming up with better ways to ship
optimization instances around. That is why IMHO, amongst other
developments we need a better, open, comprehensive standard.

> AMPL is a very nice modeling language.  <snip> 
...
> </snip> However, the real impediment for the 
> acceptance of any modeling language as such a standard is that when someone 
> formulates a problem using a modeling language, the model is their 
> intellectual property.  They are not keen on sharing that with others, 
> including vendors!  

That is not really my vision for the standard. I think of communicating
instances, not models. That distinction gets complicated once programs get
sufficiently sofisticated (especially with CLP), and that is one of the
things we need to figure out.

> So one concern is that when you propose a standard that gets closer to a 
> formulation, there may be less likelihood for people to share that standard 
> with others.
> 

That's important. I hadn't though about it that way. I hope you (Irv) can
raise a flag for that once we move to technical aspects. Do you think
block structures would make users uncomfortable? Actually I think this
point is important enough that it should be a new thread, assuming anyone
else has an opinion...

> Sure it has linear complexity.  For an LP with m rows, n columns, and nz 
> nonzeros, the format can be parsed in O(n + m + nz) operations.  This means 
> that the number of operations is some constant alpha times (n + m + nz). 
> But you're ignoring the fact that alpha is on the order of 1.0e9. (Pardon 
> my sarcasm and cynicism here).

Forgiven. I share this concern, in the cases where files would actually be
written and read. But I believe we can engineer to keep alpha pretty low.
BTW, how important is read/write time in the business environment?

> >  since we can express
> >a tremendous amount of structure using XML, we will actually produce
> >smaller files to begin with. Imagine for example building a matrix out of
> >blocks and having only one copy of the block. For an example of this see
> >my informs presentation.
> <snip></snip>
> I'm also not sure if people really "think" this way in 
> modeling.  Certainly when you write an AMPL model, the block structure is 
> not something you think about.  And would it be that easy to figure out the 
> blocks from an AMPL model?  (Maybe so - this would be interesting for 
> someone in academia to investigate).

I don't think of the format as any more useful for modeling than MPS is. I
would hope people don't think about the block structure when they are
modeling. But the structure can be useful for computers to look at, and is
something that can be extracted from the model. It is certainly nontrivial
to get it from ampl, especially after presolve, but it is nice to have it
available in the standard for future versions of ampl or anything else to
take advantage of, barring the IP issues Irv mentioned.

> >Developing a new standard means finding a cheaper, faster, easier way to
> >convey mathematical programs of as many practical categories as possible,
> >with as much structure information as possible, in an efficient manner and
> >in the context we do business and research in.
> 
> And how does a modeling language like AMPL not achieve this goal?
> 

It does, for a lot of cases. Just like ILOG's or IBM's libraries do for
some other cases. But they all solve a different problem than this
standard would. My vision of a standard is for machine <-> machine
communication. The standard would be used by people developing tools that
complement or augment ampl or ILOG or IBM software. I think there is a lot
of value to be added in that realm, but it isn't going to happen if
manipulating optimization problems is as hard as it is now.

> >assume that it were much much easier,
> >faster, and cheaper to communicate mathematical programming instances.
> >That brings two questions:
> >
> >First, do people think there would be a greater number of profitable
> >applications involving an optimization component?
> 
> I don't think so.  The barrier today is having enough people out there who 
> know how to apply optimization (i.e., do good modeling) in the first place.

I agree that people being able to apply optimization is the number one
limitation we currently have, and I hope we are both right, since I intend
to expend quite a bit of intellectual effort over the next couple decades
working on that problem. I think that a big part of solving this problem
is creating domain-specific extensions to current modeling tools. To do
that, we will often need to interface with non-optimization applications.
It better be cheap and easy to do that, and good standards are part of the
solution.

> I think the applications are there *today*.  A standard contributes very 
> little to our ability to make those applications successful.

I also think that the applications are there *today*, and that we need a
lot more than a standard to make them successful. A standard doesn't solve
all the problems, but it helps create an environment where the problems
can be solved. Without it, there are a lot of things that would be just
too cumbersome to do. A standard is not a sufficient, but a necessary
condition.

========================================================================
Leonardo B. Lopes                                       leo at iems.nwu.edu 
Ph.D. Student                                              (847)491-8470
IEMS - Northwestern University              http://www.iems.nwu.edu/~leo