Modern Computational Finance

CHAPTER 10Effective AAD and Memory Management

In this chapter, we turn the ideas and toy code from the introductory chapters into a professional, efficient AAD library.

We have seen that, to differentiate calculation code with AAD, we instantiate the calculation code with a custom type for the representation of numbers, where all arithmetic operators and mathematical functions are overloaded, so that, in addition to being evaluated, all operations are recorded on a tape. After the calculation is complete and the entire sequence of its operations is recorded, adjoints are back-propagated over the tape to produce all its differentials in constant time.

The AAD library provides the custom number type, the tape data structure, and the object that represents an operation on tape, called node, together with back-propagation utilities to propagate adjoints over the nodes on tape in the reverse order from evaluation. Calculation code must be templated on its number type. Differentiation code executes the calculation code, instantiated with the AAD number type, which records the calculation on tape, calls the back-propagation utilities to accumulate adjoints, and reads differentials on the inputs' adjoints. This is called instrumentation. The AAD library is independent from any calculation code; it is the instrumentation code that applies it to differentiate a particular calculation.

With AAD, more so than other algorithms, efficient implementation is crucial, and algorithmic correctness ...