Mathematical Foundations for Signal Processing, Communications, and Networking

3.12    Best Approximations and Orthogonal Projections

The following result plays an important role in many signal processing applications involving estimation and optimization tasks such as solving an overdetermined system of equations in the least-squares (LS) sense [10, 11].

Theorem 3.12.1. (Orthogonal Projection Theorem [6]) Let $\mathcal{V}$ be an inner-product vector space and let $\mathcal{S}$ be a subspace of it. For every vector v ∈ $\mathcal{V}$ there may exist a best approximation s ∈ $\mathcal{S}$ such that

$\Vert v-s\Vert \le \Vert v-s^{\prime }\Vert ,\forall s^{\prime }\in \mathcal{S}.$

(3.100)

This condition holds if and only if

$\left(v-s\right)\perp s^{\prime },\forall s^{\prime }\in \mathcal{S}.$

(3.101)

Moreover, if this best approximation exists, then it is unique. Now let an orthogonal basis for $\mathcal{S}$ be denoted by {s₁, s₂,…, s_m}. It can be shown that the best approximation ...