[ADOL-C] ADOLC Speed Diff 1.8 vs 2.5.2

[Kshitij Kulshreshtha]

Hello Scott,

the use of .adolcrc and increasing buffer-sizes in usrparms.h is
completely equivalent. They both change the same variables in libadolc
with .adolcrc always superseeding the values in usrparms.h

The behaviour with later iterations being slower than earlier ones makes
me wonder though. It is also worrisome that the trace created in the
first iteration gives wrong results later on, leading to wrong
minimisation, this should definitely not happen.

Please make sure that there are no adouble variables existing in program
core before the execution of trace_on() and none are left in the program
core after the execution of trace_off(). All remaining adouble variables
must be destroyed immediately before trace_off(). Do not use adoubles in
any persistent data structures.

Best regards
Kshitij

On 2014-11-20 01:06, Scott Rasmussen @ Zaita wrote:
> Hi Andrea and Kshitij,
> 
> I have tested the solutions you both offered, but they didn't improve
> the situation.
> Doing a trace_on() when a gradient is required, or first iteration only,
> causes the minimisation results to get worse, but execution time is 18
> seconds.
> Increasing the buffer size stopped the file I/O but this didn't help
> execution time.
> 
> Attached to this email are the binaries I have been using and
> configuration files to run them if you're interested.
> Below are the results of each configuration and test:
> 
> Binary: isam_adolc18.exe (ADOLC version 1.8)
> Process memory usage: 20MB
> Files created: none
> Extra notes: does run trace_on() / trace_off() on every iteration
> Execution time: 3 seconds
> 
> Binary: isam_adolc_2.5.2.exe
> Process memory usage: 8-10MB
> Files created:
> ADOLC-Locations_0.tap (0 - 10MB)
> ADOLC-Operations_0.tap (0 - 1MB)
> ADOLC-Taylors_0.tap (0kbs)
> Extra notes: first 2-3 iterations run almost instantly, then they slow
> down to 3-4 seconds each
> Execution time: 2.9 minutes
> 
> Binary: isam_increased_buffer.exe
> New buffer sizes: 134217728
> Process memory usage: 21-22MB
> Files created: none
> Extra notes: first 2-3 iterations run almost instantly, then they slow
> down to 3-4 seconds each
> Execution time: 2.8 minutes
> 
> Binary: isam_adolc_2_5_2.exe (with adolrc)
> Process memory usage: 21-22MB
> Files created: none
> New buffer values:
> "OBUFSIZE" = "512000000"
> "LBUFSIZE" = "512000000"
> "VBUFSIZE" = "512000000"
> "TBUFSIZE" = "512000000"
> Extra notes: first 2-3 iterations run almost instantly, then they slow
> down to 3-4 seconds each
> Execution time: 2.8 minutes
> 
> Binary: isam_adolc_2_5_2_trace_on_when_gradient_required.exe
> Process memory usage: 21-22MB
> Files created: None
> Extra notes: this is also using a compiled increased buffer size,
> Extra notes: first 2-3 iterations run almost instantly, then they slow
> down to 3-4 seconds each
> Extra notes: Minimisation did not match the results of the other
> binaries. They were 1998.5 and this is 2647.97 so considerably worse
> Execution time: 18 seconds
> 
> Binary: isam_adolc_2_5_2_trace_only_first_iteration.exe
> Process memory usage: 21-22MB
> Files created: None
> Extra notes: this is also using a compiled increased buffer size
> Extra notes: first 2-3 iterations run almost instantly, then they slow
> down to 3-4 seconds each
> Extra notes: Minimisation did not match the results of the other
> binaries. They were 1998.5 and this is 2647.97 so considerably worse
> Execution time: 18 seconds
> 
> Thanks heaps for your help. I hope I can get this resolved so I can make
> ADOLC the default auto-diff library for the new modelling platform.
> 
> Regards,
> Scott
> 
> On 19/11/2014 8:09 p.m., Andrea Walther wrote:
>> Hi Scott,
>>
>>> I notice it writes to files A LOT and not a huge amount of data (any
>>> flags to keep this in memory?)
>>
>> that is most probably the reason why the current version is so slow.
>> It is really important to avoid the writting on disc to get a
>> reasonable speed.
>>
>> In the directory
>>
>> .../ADOL-C/include/adolc/internal
>>
>> there is a file
>>
>> usrparms.h
>>
>> There you can increase the buffer size for
>>
>> OBUFSIZE
>> LBUFSIZE
>> VBUFSIZE
>> TBUFSIZE
>>
>> to such a large value that the files are not written on disc.
>> Alternatively you could use a
>>
>> .adolcrc
>>
>> file. I attach an example file. This would avoid the recompilation
>> if you have to change the buffer sizes.
>>
>> And finally it is important that you use
>>
>> trace_off()
>>
>> and do NOT use
>>
>> trace_off(1) or something like that
>>
>> because this would force ADOL-C to write the files on disc.
>>
>> Hope this helps. Otherwise just email again.
>>
>> Best regards
>>
>> Andrea
>>
> 
> 
> 
> _______________________________________________
> ADOL-C mailing list
> ADOL-C at list.coin-or.org
> http://list.coin-or.org/mailman/listinfo/adol-c
> 

-- 
Dr. Kshitij Kulshreshtha

Institut für Mathematik,
Universität Paderborn,
Warburger Straße 100,
33098 Paderborn.

Büro: A3.235

Privatanschrift:
Arnikaweg 62
33100 Paderborn.