[Coin-ipopt] Limit on MU

Wed Aug 3 13:57:13 EDT 2005

Andreas, Carl,

Thanks for the replies.  First let me say that the IPOPT code is of very
high quality (we've tinkered with both the Fortran and the C++ versions)
and we think it will be very helpful in our work.

Andreas - yes, the DTOL/10 limit is hard-coded into update_mu.f, and in
fact overrides a previous statement based on QMUERRFAC (which I believe
is currently prevented from affecting the algorithm):

         mumin = QTOL/(2.d0*(QMUERRFAC+1.d0))
         mumin = QTOL/10.d0

Carl - you are dead on with your observation that we set DTOL large due
to noisy function (and even noisier gradient) calculations.  By the way,
if there are any new / promising approaches to noisy optimization that
you are aware of (or working on) we'd love to hear about it.  We tried
DINFMAXTOL, but we observed that drastically reducing mumin actually
allows us to converge faster, since it impacts the iteration trajectory,
rather than serving in the limited capacity of an end criterion (unless
there's more going on than what I see in solve_barrier.F).

Here's an unrelated issue that I noticed from another post regarding the
C++ Visual Studio version.  We've been able to compile this version
successfully but had to make some adjustments.  Here's what works for us
(this comes from our VC++ project file):

			<Filter
				Name="Algorithm"
				Filter="">
				<File

RelativePath="..\..\..\Algorithm\IpAlgBuilder.cpp">
				</File>
				<File

RelativePath="..\..\..\Algorithm\IpDefaultIterateInitializer.cpp">
				</File>
				<File

RelativePath="..\..\..\Algorithm\IpFilter.cpp">
				</File>
				<File

RelativePath="..\..\..\Algorithm\IpFilterLineSearch.cpp">
				</File>
				<File

RelativePath="..\..\..\Algorithm\IpIpoptAlg.cpp">
				</File>
				<File

RelativePath="..\..\..\Algorithm\IpIpoptCalculatedQuantities.cpp">
				</File>
				<File

RelativePath="..\..\..\Algorithm\IpIpoptData.cpp">
				</File>
				<File

RelativePath="..\..\..\Algorithm\IpLoqoMuOracle.cpp">
				</File>
				<File

RelativePath="..\..\..\Algorithm\IpMonotoneMuUpdate.cpp">
				</File>
				<File

RelativePath="..\..\..\Algorithm\IpNonmonotoneMuUpdate.cpp">
				</File>
				<File

RelativePath="..\..\..\Algorithm\IpOptErrorConvCheck.cpp">
				</File>
				<File

RelativePath="..\..\..\Algorithm\IpOptProbingMuOracle.cpp">
				</File>
				<File

RelativePath="..\..\..\Algorithm\IpOrigIpoptNLP.cpp">
				</File>
				<File

RelativePath="..\..\..\Algorithm\IpOrigIterationOutput.cpp">
				</File>
				<File

RelativePath="..\..\..\Algorithm\IpProbingMuOracle.cpp">
				</File>
				<File

RelativePath="..\..\..\Algorithm\IpRestoFilterConvCheck.cpp">
				</File>
				<File

RelativePath="..\..\..\Algorithm\IpRestoIpoptNLP.cpp">
				</File>
				<File

RelativePath="..\..\..\Algorithm\IpRestoIterateInitializer.cpp">
				</File>
				<File

RelativePath="..\..\..\Algorithm\IpRestoIterationOutput.cpp">
				</File>
				<File

RelativePath="..\..\..\Algorithm\IpRestoMinC_1Nrm.cpp">
				</File>
				<File

RelativePath="..\..\..\Algorithm\IpRestoRestoPhase.cpp">
				</File>
				<File

RelativePath="..\..\..\Algorithm\IpWarmStartIterateInitializer.cpp">
				</File>
			</Filter>
			<Filter
				Name="LinalgSerial"
				Filter="">
				<File

RelativePath="..\..\..\LinAlgImpl\Serial\IpBlas.cpp">
				</File>
				<File

RelativePath="..\..\..\LinAlgImpl\Serial\IpDenseVector.cpp">
				</File>
				<File

RelativePath="..\..\..\LinAlgImpl\Serial\IpExpansionMatrix.cpp">
				</File>
				<File

RelativePath="..\..\..\LinAlgImpl\Serial\IpGenTMatrix.cpp">
				</File>
				<File

RelativePath="..\..\..\LinAlgImpl\Serial\IpSymTMatrix.cpp">
				</File>
				<File

RelativePath="..\..\..\LinAlgImpl\Serial\IpTNLPAdapter.cpp">
				</File>
			</Filter>
			<Filter
				Name="Common"
				Filter="">
				<File

RelativePath="..\..\..\Common\IpCompoundMatrix.cpp">
				</File>
				<File

RelativePath="..\..\..\Common\IpCompoundSymMatrix.cpp">
				</File>
				<File

RelativePath="..\..\..\Common\IpCompoundVector.cpp">
				</File>
				<File

RelativePath="..\..\..\Common\IpDebug.cpp">
				</File>
				<File

RelativePath="..\..\..\Common\IpDiagMatrix.cpp">
				</File>
				<File

RelativePath="..\..\..\Common\IpIdentityMatrix.cpp">
				</File>
				<File

RelativePath="..\..\..\Common\IpJournalist.cpp">
				</File>
				<File

RelativePath="..\..\..\Common\IpMatrix.cpp">
				</File>
				<File

RelativePath="..\..\..\Common\IpObserver.cpp">
				</File>
				<File

RelativePath="..\..\..\Common\IpOptionsList.cpp">
				</File>
				<File

RelativePath="..\..\..\Common\IpReferenced.cpp">
				</File>
				<File

RelativePath="..\..\..\Common\IpSumMatrix.cpp">
				</File>
				<File

RelativePath="..\..\..\Common\IpSumSymMatrix.cpp">
				</File>
				<File

RelativePath="..\..\..\Common\IpSymMatrix.cpp">
				</File>
				<File

RelativePath="..\..\..\Common\IpTaggedObject.cpp">
				</File>
				<File

RelativePath="..\..\..\Common\IpVector.cpp">
				</File>
				<File

RelativePath="..\..\..\Common\IpZeroMatrix.cpp">
				</File>
			</Filter>
			<Filter
				Name="PDFullSpace"
				Filter="">
				<File

RelativePath="..\..\..\PDSystemImpl\PDFullSpace\IpAugRestoSystemSolver.c
pp">
				</File>
				<File

RelativePath="..\..\..\PDSystemImpl\PDFullSpace\IpAugSystemSolver.hpp">
				</File>
				<File

RelativePath="..\..\..\PDSystemImpl\PDFullSpace\IpLeastSquareMults.cpp">
				</File>
				<File

RelativePath="..\..\..\PDSystemImpl\PDFullSpace\IpMa27TSolverInterface.c
pp">
				</File>
				<File

RelativePath="..\..\..\PDSystemImpl\PDFullSpace\IpMa27TSolverInterface.h
pp">
				</File>
				<File

RelativePath="..\..\..\PDSystemImpl\PDFullSpace\IpMc19TSymScalingMethod.
cpp">
				</File>
				<File

RelativePath="..\..\..\PDSystemImpl\PDFullSpace\IpPardisoSolverInterface
.cpp">
				</File>
				<File

RelativePath="..\..\..\PDSystemImpl\PDFullSpace\IpPDFullSpaceSolver.cpp"
>
				</File>
				<File

RelativePath="..\..\..\PDSystemImpl\PDFullSpace\IpStdAugSystemSolver.cpp
">
				</File>
				<File

RelativePath="..\..\..\PDSystemImpl\PDFullSpace\IpTripletHelper.cpp">
				</File>
				<File

RelativePath="..\..\..\PDSystemImpl\PDFullSpace\IpTripletToCSRConverter.
cpp">
				</File>
				<File

RelativePath="..\..\..\PDSystemImpl\PDFullSpace\IpTSymLinearSolver.cpp">
				</File>
			</Filter>
			<Filter
				Name="StdInterface"
				Filter="">
				<File

RelativePath="..\..\..\Apps\StdInterface\IpIpoptApplication.cpp">
				</File>
				<File

RelativePath="..\..\..\Apps\StdInterface\IpIpoptApplication.hpp">
				</File>
				<File

RelativePath="..\..\..\Apps\StdInterface\IpStdCInterface.cpp">
				</File>
				<File

RelativePath="..\..\..\Apps\StdInterface\IpStdFInterface.c">
				</File>
				<File

RelativePath="..\..\..\Apps\StdInterface\IpStdInterfaceTNLP.cpp">
				</File>
				<File

RelativePath="..\..\..\Apps\StdInterface\IpStdInterfaceTNLP.hpp">
				</File>
			</Filter>
		</Filter>

Regards,

--Ivan.

----------------------------
Ivan B. Oliveira
SC12-205
(408)765-0584

-----Original Message-----
From: Andreas Waechter [mailto:andreasw at watson.ibm.com] 
Sent: Wednesday, August 03, 2005 10:08 AM
To: Carl Damon Laird
Cc: Oliveira, Ivan B; coin-ipopt at list.coin-or.org
Subject: Re: [Coin-ipopt] Limit on MU

Hi again,

Reading Carl's reply I realized that the fix I described probably
doesn't
do what you want anyway.  That version of Ipopt works by solving a
sequence of barrier problems to relaxed tolerances, and the barrier
parameter for each barrier problem is proportinal to the relaxed error
tolerance.  The proportionality factor is determined by the option
"DMUERRFAC", which by default is 10.  If you choose a larger value, then
the barrier parameters for a given tolerance will be smaller, and in
particular, for the final relaxed error tolerance (which is on the order
of DTOL) the algorithm will use a smaller MU - just as you want it.
However, you will still have to modify the code at that point that I
mentioned because the factor 10 is hardcoded there (and the alg might
just
cycle for a larger value of DMUERRFAC - something that Dominique Orban
recently made me aware of...)

Regards,

Andreas

On Wed, 3 Aug 2005, Carl Damon Laird wrote:

> Hi Ivan,
>
> I assume that the reason you want a smaller mu is to produce a
solution
> that is tighter to the bound for the active variables.
>
> It has been my experience that reducing dinfmaxtol (to something like
> 1e-10) pulls things in tighter, although I have never done this with a
> large overall tolerance. Furthermore, I assume that you use a large
> tolerance because of noise in the problem (derivatives?) and this is
not
> the case with my problems.
>
> Anyway, it is something to try before modifying the code.
>
> Cheers,
>
> Carl.
>
>
>
> On Wed, 3 Aug 2005, Andreas Waechter wrote:
>
> > Hi Ivan,
> >
> >> We've been exploring IPOPT (Fortran version) for some time in
solving
> >> our nonlinear programming problem and ran into a curious behavior.
> >
> > Out of curiosity:  What kind of problems are you solving?
> >
> >> We're in the unfortunate position of having to set DTOL rather
large and
> >> we've noticed that the barrier parameter is hard-bounded by
DTOL/10.  In
> >> your paper, you explain: "the update (7) does not allow the rule to
> >> become smaller than necessary given the desired tolerance
> >> \epsilon_{tol}, thus avoiding numerical difficulties at the end of
the
> >> optimization procedure."  We are unclear on what the specific
numerical
> >> difficulties would be for this primal-dual formulation of the
problem.
> >> Are there any results that have shown what the 'necessary' value of
MU
> >> should be in relation to the overall tolerance?
> >
> > If you would choose MU to be on the order of, say, 1e-16, you might
run
> > into problems because your slacks and bound multiplier values become
very
> > small (on the order of machine precision).  Therefore, the idea was
that
> > the mimimal value of the barrier parameter should be tied to the
specified
> > error tolerance.
> >
> > If you specify that you only want to solve the problem to a
tolerance of,
> > say, 1e-3, the lower bound on MU will be small enough so that your
desired
> > tolerance can be met, but not much smaller.  If you want to have the
> > algorithm use a smaller value of MU (and therefore solve the
> > complementarity condition more tightly than the other optimality
> > conditions), you can change line 96 of update_mu.f accordingly.
> >
> >> Since I have you attention I can't resist asking if there are any
> >> updates on when the C++ alpha version will be released?
> >
> > We hope to be ready to release the C++ version in about three weeks.
> > Therefore, we are not making any more changes to the Fortran version
> > (except for bug fixes).  The new version will not have all the
"features"
> > of the Fortran version (e.g., there will be no reduced-space option
- and
> > the limitied-memory quasi-Newton options for the full space will be
added
> > a little bit later).
> >
> > Hope this helps,
> >
> > Andreas
> >
> > _______________________________________________
> > Coin-ipopt mailing list
> > Coin-ipopt at list.coin-or.org
> > http://list.coin-or.org/mailman/listinfo/coin-ipopt
> >
> >
> _______________________________________________
> Coin-ipopt mailing list
> Coin-ipopt at list.coin-or.org
> http://list.coin-or.org/mailman/listinfo/coin-ipopt
>