Lecture 24
Orthogonal Complements

We want to generalize our procedure of decomposing a vector into one parallel to a given line and one perpendicular to that line, by replacing the line (really a direction determined by a single vector) with any nonempty collection of vectors.

Definition. The orthogonal complement of a nonempty subset S of Rⁿ is the set of all vectors in Rⁿ that are orthogonal to every vector in S:

Notice that the orthogonal complement of S is automatically a subspace of Rⁿ, since it is easily seen to be closed under addition and scalar multiplication, and to contain 0.

Notice that the orthogonal complement of a subspace of Rⁿ is the same as the orthogonal complement of a basis for that subspace.

Example

Notice that we can view the orthogonal complement of a finite set of vectors as defined by a linear system. The corresponding matrix is the one whose rows are given by the vectors in S, and the orthogonal complement is then clearly its null space.

In particular, if the rows of A are a basis for a subspace W of Rⁿ which has dimension k, then the rank of A is k and the nullity of A is n - k. This means that the dimension of the row space and the dimension of the space perpendicular to it automatically add up to the total dimension n.

Example

Now we can consider splitting vectors in R ⁿ into pieces in a given subspace and pieces in the orthogonal complement of that subspace.

Orthogonal Decomposition Theorem Let W be a subspace of Rⁿ. Then for any vector v in Rⁿ, there exists a unique vector w in W, and a unique vector z in W^{^}, such that v = w + z. The vector w is called the orthogonal projection of v onto W.

Indeed, we let {w₁, . . . , w_k} be an orthonormal basis for W, and we want to find the appropriate coefficients to set w = c₁w₁ + · · · + c_kw_k. But if we set z = (v - w), then the requirement thatz be perpendicular to W, that is, (v - w)·w_j = 0 for each choice of j, 1 < j < k, will determine those coefficients. These equations are just (v - (c₁w₁ + · · · + c_kw_k)) ·w_j = 0, which by orthogonality are v·w_j - c_j w_j ·w_j = 0. Then by normality, this means that each c_j = v·w_j, and the vectors w and z = (v - w) are uniquely determined.

In fact, the vector w is the closest to the original vector v, among all vectors in W.

Closest Vector Property
Let W be a subspace of Rⁿ, and let v be a vector in Rⁿ. Then the vector closest to v, among all the vectors in W, is the orthogonal projection of v onto W.

Indeed, let w denote the orthogonal projection of v onto W. We have, for any w' in W,

Now, (v - w) = z is orthogonal to W, and (w - w') is in W, so is orthogonal to z. By the Pythagorean Theorem, then, ||v - w'||² = ||v - w||² + ||w - w'||² > ||v - w||², and the result follows.

250 Lecture Index