[Coin-discuss] what does it mean to peer review software?

[Ted Ralphs]

Robin Lougee-Heimer wrote:
> 
> Hey there,
> At the recent  INFORMS meeting, the question came up of what it means to
> "peer review" software.   This is especially important for people in the
> tenure process who would like to contribute code to COIN-OR (or other open
> source projects) but need credit towards tenure.

Yes :).

> Not being in academics,  I have talked to a number of deans/deparment heads
> (e.g., John Birge U Michigan;  Russell Barton Penn State; David Ryan  Univ.
> of Auckland, New Zealand; and a few others) investigating this "tenure"
> mystery.   Their  general opinion was that deans do not really matter in
> the tenure process, what's really important is the supporting letters from
> colleagues and the department head's opinion.

This may be true at many schools, but it's not the message I'm getting. Here, 
I'm told the outside letters are generally taken with a grain of salt because 
they almost never say anything negative. One's external colleagues have no 
vested interest in the process and may want to avoid making enemies. They may 
even stand to benefit by gaining a tenured colleague at another school. As for 
department heads, the same can be true for them. If the department head were not 
fully in support of a candidate, the candidate would probably not have made it 
to the tenure decision to begin with. The problems seem to occur outside the 
department with faculty in other disciplines who want objective evidence of the 
import of the candidate's work

At Lehigh, our current dean controls the tenure criteria pretty tightly. He 
publishes a set of guidelines regarding what factors are to be and are not to be 
considered in the tenure decision. This set of guidelines is voted on by the 
faculty, but the dean has a heavy influence. Once the guidelines are in place, 
they inform the process substantially. Right now, our guidelines state that 
software is considered a "publication" of sorts, but you have to somehow show 
its merit. It's not specified stated how this should be done, however. At 
INFORMS, some of us discussed ways in which we could support the process of 
establishing merit. In a follow-up e-mail, I'll try to document some of these ideas.

> As it will be posted (soon :-) in the coin-announce mailing list, the
> INFORMS board unanimously voted to host COIN-OR on their servers.  This
> transfer will be done through a soon-to-be created nonprofit organization
> which will (should?) have by-laws, which should include some notion of
> "peer reveiw".    

I believe we will be required to have by-laws (every corporation has by-laws, 
right?), but I would be surprised if they say anything about the peer-review 
process. Most likely, we will need some generic set of rules about how the board 
gets elected, how decisions are made, etc, etc, although I could be wrong. At 
any rate, I don't think we necessarily have to settle on a notion of peer-review 
to start the process of becoming a non-profit. But I still think it's an 
important topic for us to discuss while we figure out how to reorganize ourselves.

> So, to figure out how to define "peer reveiwed software",
> it would be very important to get as many opinions  as possible.

I will start asking around.

> Please query the folks at your institution, and post their opinions here.
> 
> Thanks,
> Robin and Laci

Thanks for spearheading this effort!

> 
> ----------------------------------------------------------------------------------
> 
> Robin Lougee-Heimer
> IBM TJ Watson Research Center
> ph: 914-945-3032   fax: 914-945-3434
> robinlh at us.ibm.com
> http://www.coin-or.org
> 
> _______________________________________________
> Coin-discuss mailing list
> Coin-discuss at www-124.ibm.com
> http://www-124.ibm.com/developerworks/oss/mailman/listinfo/coin-discuss



-- 
Dr. Ted Ralphs
Assistant Professor
Industrial and Systems Engineering
Lehigh University
(610)758-4784
tkralphs at lehigh.edu
www.lehigh.edu/~tkr2