Orthogonality

In this section we define a couple more operations with vectors, and prove a few theorems. At first blush these definitions and results will not appear central to what follows, but we will make use of them at key points in the remainder of the course (such as , ). Because we have chosen to use $\complexes$ as our set of scalars, this subsection is a bit more, uh, complex than it would be for the real numbers. We will explain as we go along how things get easier for the real numbers ${\mathbb R}$. If you have not already, now would be a good time to review some of the basic properties of arithmetic with complex numbers described in . With that done, we can extend the basics of complex number arithmetic to our study of vectors in $\complex{m}$.

1. \set{\colvector{1\\-1\\2},\,\colvector{5\\3\\-1},\,\colvector{8\\4\\-2}}
2. What is the distinction between an orthogonal set and an orthonormal set?
3. What is nice about the output of the Gram-Schmidt process?

We know how the addition and multiplication of complex numbers is employed in defining the operations for vectors in $\complex{m}$ ( and ). We can also extend the idea of the conjugate to vectors.

With this definition we can show that the conjugate of a column vector behaves as we would expect with regard to vector addition and scalar multiplication.

These two theorems together tell us how we can push complex conjugation through linear combinations.

This operation is a bit different in that we begin with two vectors but produce a scalar. Computing one is straightforward.

In the case where the entries of our vectors are all real numbers (as in the second part of ), the computation of the inner product may look familiar and be known to you as a dot product or scalar product. So you can view the inner product as a generalization of the scalar product to vectors from $\complex{m}$ (rather than ${\mathbb R}^m$).

Note that we have chosen to conjugate the entries of the first vector listed in the inner product, while it is almost equally feasible to conjugate entries from the second vector instead. In particular, prior to Version 2.90, we did use the latter definition, and this has now changed to the former, with resulting adjustments propogated up through (only). However, conjugating the first vector leads to much nicer formulas for certain matrix decompositions and also shortens some proofs.

There are several quick theorems we can now prove, and they will each be useful later.

Condensed — the full section is in Beezer, A First Course in Linear Algebra.

If treating linear algebra in a more geometric fashion, the length of a vector occurs naturally, and is what you would expect from its name. With complex numbers, we will define a similar function. Recall that if $c$ is a complex number, then $\modulus{c}$ denotes its modulus ().

Computing a norm is also easy to do.

Notice how the norm of a vector with real number entries is just the length of the vector. Inner products and norms are related by the following theorem.

When our vectors have entries only from the real numbers says that the dot product of a vector with itself is equal to the length of the vector squared.

Notice that contains three implications: \vect{u}\in\complex{m}&\Rightarrow\innerproduct{\vect{u}}{\vect{u}}\geq 0\\ \vect{u}=\zerovector&\Rightarrow\innerproduct{\vect{u}}{\vect{u}}=0\\ \innerproduct{\vect{u}}{\vect{u}}=0&\Rightarrow\vect{u}=\zerovector

The results contained in are summarized by saying the inner product is positive definite.

Condensed — the full section is in Beezer, A First Course in Linear Algebra.

Orthogonal is a generalization of perpendicular. You may have used mutually perpendicular vectors in a physics class, or you may recall from a calculus class that perpendicular vectors have a zero dot product. We will now extend these ideas into the realm of higher dimensions and complex scalars.

We extend this definition to whole sets by requiring vectors to be pairwise orthogonal. Despite using the same word, careful thought about what objects you are using will eliminate any source of confusion.

We now define the prototypical orthogonal set, which we will reference repeatedly.

Notice that $\vect{e}_j$ is identical to column $j$ of the $m\times m$ identity matrix $I_m$ () and is a pivot column for $I_m$, since the identity matrix is in reduced row-echelon form. These observations will often be useful. We will reserve the notation $\vect{e}_i$ for these vectors. It is not hard to see that the set of standard unit vectors is an orthogonal set.

Condensed — the full section is in Beezer, A First Course in Linear Algebra.

The Gram-Schmidt Procedure is really a theorem. It says that if we begin with a linearly independent set of $p$ vectors, $S$, then we can do a number of calculations with these vectors and produce an orthogonal set of $p$ vectors, $T$, so that $\spn{S}=\spn{T}$. Given the large number of computations involved, it is indeed a procedure to do all the necessary computations, and it is best employed on a computer. However, it also has value in proofs where we may on occasion wish to replace a linearly independent set by an orthogonal set.

This is our first occasion to use the technique of mathematical induction for a proof, a technique we will see again several times, especially in . So study the simple example described in first.

One final definition related to orthogonal vectors.

Once you have an orthogonal set, it is easy to convert it to an orthonormal set multiply each vector by the reciprocal of its norm, and the resulting vector will have norm 1. This scaling of each vector will not affect the orthogonality properties (apply ).

We will see orthonormal sets again in . They are intimately related to unitary matrices () through . Some of the utility of orthonormal sets is captured by in . Orthonormal sets appear once again in where they are key in orthonormal diagonalization.

Condensed — the full section is in Beezer, A First Course in Linear Algebra.