Matrix multiplication

We know how to add vectors and how to multiply them by scalars. Together, these operations give us the possibility of making linear combinations. Similarly, we know how to add matrices and how to multiply matrices by scalars. In this section we mix all these ideas together and produce an operation known as matrix multiplication. This will lead to some results that are both surprising and central. We begin with a definition of how to multiply a vector by a matrix.

1. \begin{bmatrix} 2 & 3 & -1 & 0\\ 1 & -2 & 7 & 3\\ 1 & 5 & 3 & 2\\ \end{bmatrix} &&\text{with}&& \colvector{2\\-3\\0\\5}
2. \begin{bmatrix} 2 & 3 & -1 & 0\\ 1 & -2 & 7 & 3\\ 1 & 5 & 3 & 2\\ \end{bmatrix} && \begin{bmatrix} 2 & 6\\ -3 & -4\\ 0 & 2\\ 3 & -1\\ \end{bmatrix}
3. 2x_1 + 3x_2 - x_3 &= 0\\ x_1 + 2x_2 + x_3 &= 3\\ x_1 + 3x_2 + 3x_3 &= 7

We have repeatedly seen the importance of forming linear combinations of the columns of a matrix. As one example of this, the oft-used , said that every solution to a system of linear equations gives rise to a linear combination of the column vectors of the coefficient matrix that equals the vector of constants. This theorem, and others, motivate the following central definition.

So, the matrix-vector product is yet another version of multiplication, at least in the sense that we have yet again overloaded juxtaposition of two symbols as our notation. Remember your objects, an $m\times n$ matrix times a vector of size $n$ will create a vector of size $m$. So if $A$ is rectangular, then the size of the vector changes. With all the linear combinations we have performed so far, this computation should now seem second nature.

We can now represent systems of linear equations compactly with a matrix-vector product () and column vector equality (). This finally yields a very popular alternative to our unconventional $\linearsystem{A}{\vect{b}}$ notation.

Condensed — the full section is in Beezer, A First Course in Linear Algebra.

We now define how to multiply two matrices together. Stop for a minute and think about how you might define this new operation.

Many books would present this definition much earlier in the course. However, we have taken great care to delay it as long as possible and to present as many ideas as practical based mostly on the notion of linear combinations. Towards the conclusion of the course, or when you perhaps take a second course in linear algebra, you may be in a position to appreciate the reasons for this. For now, understand that matrix multiplication is a central definition and perhaps you will appreciate its importance more by having saved it for later.

Is this the definition of matrix multiplication you expected? Perhaps our previous operations for matrices caused you to think that we might multiply two matrices of the same size, entry-by-entry? Notice that our current definition uses matrices of different sizes (though the number of columns in the first must equal the number of rows in the second), and the result is of a third size. Notice too in the previous example that we cannot even consider the product $BA$, since the sizes of the two matrices in this order are not right.

But it gets weirder than that. Many of your old ideas about multiplication will not apply to matrix multiplication, but some still will. So make no assumptions, and do not do anything until you have a theorem that says you can. Even if the sizes are right, matrix multiplication is not commutative order matters.

Condensed — the full section is in Beezer, A First Course in Linear Algebra.

While certain natural properties of multiplication do not hold, many more do. In the next subsection, we will state and prove the relevant theorems. But first, we need a theorem that provides an alternate means of multiplying two matrices. In many texts, this would be given as the definition of matrix multiplication. We prefer to turn it around and have the following formula as a consequence of our definition. It will prove useful for proofs of matrix equality, where we need to examine products of matrices, entry-by-entry.

is the way many people compute matrix products by hand. It will also be very useful for the theorems we are going to prove shortly. However, the definition () is frequently the most useful for its connections with deeper ideas like the null space and the upcoming column space.

Condensed — the full section is in Beezer, A First Course in Linear Algebra.

In this subsection, we collect properties of matrix multiplication and its interaction with the zero matrix (), the identity matrix (), matrix addition (), scalar matrix multiplication (), the inner product (), conjugation (), and the transpose (). Whew! Here we go. These are great proofs to practice with, so try to concoct the proofs before reading them, they will get progressively more complicated as we go.

It is this theorem that gives the identity matrix its name. It is a matrix that behaves with matrix multiplication like the scalar 1 does with scalar multiplication. To multiply by the identity matrix is to have no effect on the other matrix.

Since says matrix multipication is associative, it means we do not have to be careful about the order in which we perform matrix multiplication, nor how we parenthesize an expression with just several matrices multiplied togther. So this is where we draw the line on explaining every last detail in a proof. We will frequently add, remove, or rearrange parentheses with no comment. Indeed, I only see about a dozen places where is cited in a proof. You could try to count how many times we avoid making a reference to this theorem.

Condensed — the full section is in Beezer, A First Course in Linear Algebra.

The adjoint of a matrix has a basic property when employed in a matrix-vector product as part of an inner product. At this point, you could even use the following result as a motivation for the definition of an adjoint.

Sometimes a matrix is equal to its adjoint (), and these matrices have interesting properties. One of the most common situations where this occurs is when a matrix has only real number entries. Then we are simply talking about symmetric matrices (), so you can view this as a generalization of a symmetric matrix.

Again, the set of real matrices that are Hermitian is exactly the set of symmetric matrices. In we will uncover some amazing properties of Hermitian matrices, so when you get there, run back here to remind yourself of this definition. Further properties will also appear in . Right now we prove a fundamental result about Hermitian matrices, matrix vector products and inner products. As a characterization, this could be employed as a definition of a Hermitian matrix and some authors take this approach.

So, informally, Hermitian matrices are those that can be tossed around from one side of an inner product to the other with reckless abandon. We will see later what this buys us.

Condensed — the full section is in Beezer, A First Course in Linear Algebra.