[CppAD] Automatic differentiation of tanh fails for some values

[Robert P. Goddard]

The solution for tanh overflow is simple: For x > 0.5 (or thereabouts),
switch over to this formula instead of sinh/cosh:

tanh(x) = (1 - exp(-2x)) / (1 + exp(-2x))

The C99 standard (inherited by the new C++ standard) says exp should
underflow to zero for large negative arguments, which is just what you want.

It's best NOT to use this formula for small x because the roundoff error
in the numerator gets large as exp(-2x) gets close to 1.0. The
power-series formula for tanh converges for |2x| < pi/2 or |x|<pi/4, so
it's probably best to use it up to somewhat less than that. Assuming
your sinh implementation uses a power series for small arguments, you
might as well use sinh/cosh there. But if your sinh is implemented as
(exp(x)-exp(-x))/2, that has the same round-off problem at small x, so
you might as well use the tanh formula above everywhere.

C99 also has a new expm1 function for exp(x)-1, with better accuracy for
small x. If you use that to implement the tanh formula above, it would
have good accuracy everywhere. But then CppAD would need to know its
derivative -- and if you do that, you might as well teach CppAD about
the tanh derivative and bypass all of this.

Bob Goddard
-------------- next part --------------
A non-text attachment was scrubbed...
Name: robert_goddard.vcf
Type: text/x-vcard
Size: 384 bytes
Desc: not available
Url : http://list.coin-or.org/pipermail/cppad/attachments/20110715/4612c6f0/attachment.vcf