Theoretical Foundations of Functional Data Analysis, with an Introduction to Linear Operators

Chapter 4Compact operators and singular value decomposition

In this chapter, we continue the discussion of operators that was begun in Chapter 3. However, in doing so, we will narrow our focus to the special case of operators that are compact in a sense to be described shortly. When working with Hilbert spaces, this type of operator can be approximated by finite-dimensional operators and, as a result, exhibits similar properties to those we are familiar with from the study of matrices. Not surprisingly, it is compact operators that are pervasive throughout statistics, in general, and fda, in particular.

We begin in Section 4.1 with a general treatment of compact operators. Then, we specialize again; this time to the case of operators between Hilbert spaces in Sections 4.2 and 4.3 and derive both eigenvalue and singular value decompositions (svds) for this setting.

Within the class of compact operators on Hilbert spaces, Hilbert–Schmidt and trace-class operators are of special interest due, in part, to the rapid convergence of their optimal finite-dimensional approximations. Accordingly, we investigate the properties of these operator classes in some depth in Sections 4.4 and 4.5. In functional data, integral operators are especially relevant. A key result for this type of operator is Mercer's Theorem that uses the eigenvalue–eigenvector decomposition of an integral operator to obtain a corresponding series expansion for the operator's kernel. This latter series is very important ...