Four Subsets

There are four natural subsets associated with a matrix. We have met three already: the null space, the column space and the row space. In this section we will introduce a fourth, the left null space. The objective of this section is to describe one procedure that will allow us to find linearly independent sets that span each of these four sets of column vectors. Along the way, we will make a connection with the inverse of a matrix, so will tie together most all of this chapter (and the entire course so far).

1. A= \begin{bmatrix} 2 & 1 & -3 & 4\\ -1 & -1 & 2 & -1\\ 0 & -1 & 1 & 2 \end{bmatrix}
2. A= \begin{bmatrix} -9 & 5 & -3 \\ 2 & -1 & 1 \\ -5 & 3 & -1 \end{bmatrix}
3. Why is a great conclusion to ?

The left null space will not feature prominently in the sequel, but we can explain its name and connect it to row operations. Suppose $\vect{y}\in\lns{A}$. Then by , $\transpose{A}\vect{y}=\zerovector$. We can then write \transpose{\zerovector} &=\transpose{\left(\transpose{A}\vect{y}\right)} &&\\ &=\transpose{\vect{y}}\transpose{\left(\transpose{A}\right)} &&\\ &=\transpose{\vect{y}}A &&

The product $\transpose{\vect{y}}A$ can be viewed as the components of $\vect{y}$ acting as the scalars in a linear combination of the rows of $A$. And the result is a row vector, $\transpose{\zerovector}$ that is totally zeros. When we apply a sequence of row operations to a matrix, each row of the resulting matrix is some linear combination of the rows. These observations tell us that the vectors in the left null space are scalars that record a sequence of row operations that result in a row of zeros in the row-reduced version of the matrix. We will see this idea more explicitly in the course of proving .

Condensed — the full section is in Beezer, A First Course in Linear Algebra.

We have three ways to build the column space of a matrix. First, we can use just the definition, , and express the column space as a span of the columns of the matrix. A second approach gives us the column space as the span of some of the columns of the matrix, and additionally, this set is linearly independent (). Finally, we can transpose the matrix, row-reduce the transpose, kick out zero rows, and write the remaining rows as column vectors. and tell us that the resulting vectors are linearly independent and their span is the column space of the original matrix.

We will now demonstrate a fourth method by way of a rather complicated example. Study this example carefully, but realize that its main purpose is to motivate a theorem that simplifies much of the apparent complexity. So other than an instructive exercise or two, the procedure we are about to describe will not be a usual approach to computing a column space.

This example motivates the remainder of this section, so it is worth careful study. You might attempt to mimic the second approach with the coefficient matrices of and . We will see shortly that the matrix $L$ contains more information about $A$ than just the column space.

Sage can very nearly reproduce the reduced row-echelon form we obtained from the augmented matrix with variables present in the final column. The first line below is a bit of advanced Sage. It creates a number system R mixing the rational numbers with the variables b1 through b6. It is not important to know the details of this construction right now. B is the reduced row-echelon form of A, as computed by Sage, where we have displayed the last column separately so it will all fit. You will notice that B is different than what we used in , where all the differences are in the final column.

However, we can perform row operations on the final two rows of B to bring Sage's result in line with what we used above for the final two entries of the last column, which are the most critical. Notice that since the final two rows are almost all zeros, any sequence of row operations on just these two rows will preserve the zeros (and we need only display the final column to keep track of our progress).

Notice that the last two entries of the final column now have just a single b1 and a single B2. We could continue to perform more row operations by hand, using the last two rows to progressively eliminate b1 and b2 from the other four expressions of the last column. Since the two last rows have zeros in their first five entries, only the entries in the final column would change. You will see that much of this section is about how to automate these final calculations.

Condensed — the full section is in Beezer, A First Course in Linear Algebra.

The final matrix that we row-reduced in should look familiar in most respects to the procedure we used to compute the inverse of a nonsingular matrix, . We will now generalize that procedure to matrices that are not necessarily nonsingular, or even square. First a definition.

Notice that in the case where $A$ is a nonsingular matrix we know that the reduced row-echelon form of $A$ is the identity matrix (), so $B=I_n$. Then the second conclusion above says $JA=B=I_n$, so $J$ is the inverse of $A$. Thus this theorem generalizes , though the result is a left-inverse of $A$ rather than a right-inverse.

The third conclusion of is the most telling. It says that $\vect{x}$ is a solution to the linear system $\linearsystem{A}{\vect{y}}$ if and only if $\vect{x}$ is a solution to the linear system $\linearsystem{B}{J\vect{y}}$. Or said differently, if we row-reduce the augmented matrix $\augmented{A}{\vect{y}}$ we will get the augmented matrix $\augmented{B}{J\vect{y}}$. The matrix $J$ tracks the cumulative effect of the row operations that converts $A$ to reduced row-echelon form, here effectively applying them to the vector of constants in a system of equations having $A$ as a coefficient matrix. When $A$ row-reduces to a matrix with zero rows, then $J\vect{y}$ should also have zero entries in the same rows if the system is to be consistent.

Condensed — the full section is in Beezer, A First Course in Linear Algebra.

With all the preliminaries in place we can state our main result for this section. In essence this result will allow us to say that we can find linearly independent sets to use in span constructions for all four subsets (null space, column space, row space, left null space) by analyzing only the extended echelon form of the matrix, and specifically, just the two submatrices $C$ and $L$, which will be ripe for analysis since they are already in reduced row-echelon form ().

The first two conclusions of this theorem are nearly trivial. But they set up a pattern of results for $C$ that is reflected in the latter two conclusions about $L$. In total, they tell us that we can compute all four subsets just by finding null spaces and row spaces. This theorem does not tell us exactly how to compute these subsets, but instead simply expresses them as null spaces and row spaces of matrices in reduced row-echelon form without any zero rows ($C$ and $L$). A linearly independent set that spans the null space of a matrix in reduced row-echelon form can be found easily with . It is an even easier matter to find a linearly independent set that spans the row space of a matrix in reduced row-echelon form with , especially when there are no zero rows present. So an application of is typically followed by two applications each of and .

The situation when $r=m$ deserves comment, since now the matrix $L$ has no rows. What is $\csp{A}$ when we try to apply and encounter $\nsp{L}$? One interpretation of this situation is that $L$ is the coefficient matrix of a homogeneous system that has no equations. How hard is it to find a solution vector to this system? Some thought will convince you that any proposed vector will qualify as a solution, since it makes all of the equations true. So every possible vector is in the null space of $L$ and therefore $\csp{A}=\nsp{L}=\complex{m}$. OK, perhaps this sounds like some twisted argument from Alice in Wonderland. Let us try another argument that might solidly convince you of this logic.

If $r=m$, when we row-reduce the augmented matrix of $\linearsystem{A}{\vect{b}}$ the result will have no zero rows, and the first $n$ columns will all be pivot columns, leaving none for the final column, so by the system will be consistent. By , $\vect{b}\in\csp{A}$. Since $\vect{b}$ was arbitrary, every possible vector is in the column space of $A$, so we again have $\csp{A}=\complex{m}$. The situation when a matrix has $r=m$ is known by the term full rank, and in the case of a square matrix coincides with nonsingularity (see ).

We will now illustrate with a few examples.

The next example is just a bit different since the matrix has more rows than columns, and a trivial null space.

Condensed — the full section is in Beezer, A First Course in Linear Algebra.