[Ipopt] improving IPOPT speed with Algorithmic DifferentiationTheory

[Sebastian Walter]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello Andreas, hello Andrey,

thanks for your detailed replies :)
I think we are talking about different things:
Caching and Algorithmic Differentiation (AD)

My question was actually about AD. But now that you explained the
caching in more detail, I am not so sure I had fully understood the idea
of the new_x flag.

1) === The caching
Wasn't there a remark in the IPOPT documentation that IPOPT does this
caching automatically? I'm not sure what calculate() and cache()
really do.
Did I get this right?
- -- The caching is performed automatically by IPOPT for each f, grad_f,
g, jac_g separately.
- -- but sometimes it is very cheap to compute f,grad_f, g, jac_g,... at
once. To do this all-at-once when a new x has been computed, the new_x
flag has been introduced.

2) the Algorithmic Differentiation Theory

Well, what I meant to say was:
In SNOPT you specify the objective function similar to this:

objective_function( int mode, double *x, double *f, double *grad_f);

So there is only one function for BOTH the gradient and the function.
If mode = 0: only f is evaluated
if mode = 1: only grad_f is evaluated
if mode = 2: both grad_f and f are evaluated

SNOPT knows when it is going to need both grad_f and f and then calls
the objective_function with mode==2.
Since during the computation of grad_f one always implicitly computes f,
one can save some function calls.

Now I don't really know how IPOPT works internally, but there should be
many cases when at a point x both the function and its gradient are
needed. E.g. in the first step of the line search.
And this is what I also observed when I kept track of the updates of x.
The call sequence looks like this:

eval_f(x_k)
eval_grad_f(x_k)
eval_f(x_k+1)
eval_grad_f(x_k+1)

one could save the function calls:
eval_f_and_grad_f_at_once(x_k)
eval_f_and_grad_f_at_once(x_k+1)

Our function evaluations are pretty expensive, that's why it would be
nice to save those redundant calls.

What I was trying to explain before was that I tried to use new_x as a
workaround, but i realized that my workaround would only work when the
call sequence looked like

eval_grad_f(x_k)
eval_f(x_k)
eval_grad_f(x_k+1)
eval_f(x_k+1)

so it's the wrong order.











-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.4-svn0 (GNU/Linux)
Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org

iD8DBQFI02Yu9PBA5IG0h0ARAsaeAJ9Pammbqay8EyagLWFHpJlrWPxY5gCfSRiP
uWjZT/jmE7E9zJU7I5c9gs0=
=lm6O
-----END PGP SIGNATURE-----