Lecture 23
Orthogonal Vectors

The standard unit vectors in Rⁿ are the keys to relating algebraic notions to geometric ones. Their algebra is particularly simple because they are mutually perpendicular (and because each has length one). When dealing with subspaces of Rⁿ, it is useful to find similar collections of vectors.

Definition. A nonempty subset of nonzero vectors in Rⁿ is called an orthogonal set if every pair of distinct vectors in the set is orthogonal.

Examples

Orthogonal sets are automatically linearly independent.

Theorem Any orthogonal set of vectors is linearly independent.

To see this result, suppose that v₁, . . ., v_k are in this orthogonal set, and there are constants c₁, . . ., c_k such that c₁ v₁ + · · · + c_k v_k = 0. For any j between 1 and k, take the dot product of v_j with both sides of this equation. We obtain c_j ||v_j ||² = 0, and since v_j is not 0 (otherwise the set could not be orthogonal), this forcesc_j = 0. Thus the only linear combinations of vectors in the set which equal the 0 vector are those in which all of the coefficients are zero, which means that the set is linearly independent.

Definition. If V is a subspace of Rⁿ and S is an orthogonal set which spans V (and hence is a basis for V ), S is called an orthogonal basis for V.

Now, if {v₁, . . ., v_k } is an orthogonal basis for V and v is any element of V, we know that we can write

for some choice of coefficients. The advantage to orthogonality is that the coefficients are easy to determine. Again, by taking the dot product of v_j with both sides of this equation we obtain v·v_j = c_j ||v_j ||² and therefore c_j = v· v_j /||v_j ||² . Thus we have

Theorem (Representation of Vectors in terms of Orthogonal Bases)
Let {v₁, . . ., v_k} be an orthogonal basis for a subspace V of Rⁿ, and let v be any vector in V. Then v = (v·v₁ / ||v₁||² )v₁ + · · · + (v·v_k / ||v_k||² )v_k.

Examples

Orthogonal bases are computationally easier to deal with than general bases. Is it true that every subspace of Rⁿ has an orthogonal basis? The answer is yes, and the reason is that, starting with any basis, we can construct an orthogonal one via the following algorithm. The algorithm is derived by following the same procedure we used to find the decomposition of a vector into one parallel to a given line and one perpendicular to that line; now we decompose successive vectors into ones in the span of an already constructed subspace and one perpendicular to that subspace.

Theorem (Gram-Schmidt Algorithm) Let {u₁, . . ., u_k} be a linearly independent subset of Rⁿ. Then the set {v₁, . . ., v_k} constructed as follows is orthogonal, and has the same span as the original set:
set v₁ = u₁ , and having constructed v₁, . . ., v _{j -1}, set

Indeed, the set {v₁, . . ., v_k} has the same span as {u₁, . . ., u_k}, because each element in it is a linear combination of the corresponding element of the original set (with nonzero coefficient) and its predecessors. Assuming we have already proved that {v₁, . . ., v_j-1} is orthogonal, then {v₁, . . ., v_j }is orthogonal because for i < j we have

By induction, then, the set {v₁ ,. . ., v_k} is orthogonal.

Example

It is convenient to deal with orthogonal vectors each of which has norm 1. Such vectors are called unit vectors. Notice that any orthogonal set can be transformed into one in which each vector has norm 1, by simply dividing each vector by its norm - the orthogonality is unaffected, and the new norms are all 1. An orthogonal set of unit vectors is called an orthonormal basis, and the Gram-Schmidt procedure and the earlier representation theorem yield the following result.

Theorem
Every subspace W of Rⁿ has an orthonormal basis. If {w₁, . . ., w_k} is an orthonormal basis for W, and v is a vector in W, then the coordinates of v with respect to the basis are given by dot products:

250 Lecture Index