Linear Independence

Linear independence is one of the most fundamental conceptual ideas in linear algebra, along with the notion of a span. So this section, and the subsequent , will explore this new idea.

1. S=\set{\colvector{1\\2\\-1},\,\colvector{3\\-4\\2},\,\colvector{4\\-2\\1}}
2. S=\set{\colvector{1\\-1\\0},\,\colvector{3\\2\\2},\,\colvector{4\\3\\-4}}
3. \begin{bmatrix} 1 &3 &4\\ -1 &2 &3\\ 0 &2 &-4 \end{bmatrix}

Determine if the sets of vectors in Exercises C20C25 are linearly independent or linearly dependent. When the set is linearly dependent, exhibit a nontrivial relation of linear dependence.

tells us that a solution to a homogeneous system of equations is a linear combination of the columns of the coefficient matrix that equals the zero vector. We used just this situation to our advantage (twice!) in where we reduced the set of vectors used in a span construction from four down to two, by declaring certain vectors as surplus. The next two definitions will allow us to formalize this situation.

Notice that a relation of linear dependence is an equation. Though most of it is a linear combination, it is not a linear combination (that would be a vector). Linear independence is a property of a set of vectors. It is easy to take a set of vectors, and an equal number of scalars, all zero, and form a linear combination that equals the zero vector. When the easy way is the only way, then we say the set is linearly independent. Here are a couple of examples.

and relied on solving a homogeneous system of equations to determine linear independence. We can codify this process in a time-saving theorem.

Since is an equivalence, we can use it to determine the linear independence or dependence of any set of column vectors, just by creating a matrix and analyzing the row-reduced form. Let us illustrate this with two more examples.

As an equivalence, gives us a straightforward way to determine if a set of vectors is linearly independent or dependent.

Review and . They are very similar, differing only in the last two slots of the third vector. This resulted in slightly different matrices when row-reduced, and slightly different values of $r$, the number of nonzero rows. Notice, too, that we are less interested in the actual solution set, and more interested in its form or size. These observations allow us to make a slight improvement in .

The situation in is slick enough to warrant formulating as a theorem.

Condensed — the full section is in Beezer, A First Course in Linear Algebra.

We will now specialize to sets of $n$ vectors from $\complex{n}$. This will put off-limits, while will involve square matrices. Let us begin by contrasting and .

That and have opposite properties for the columns of their coefficient matrices is no accident. Here is the theorem, and then we will update our equivalences for nonsingular matrices, .

Here is the update to .

As will be our habit, we can illustrate properties of nonsingular matrices with random square matrices. Review for a refresher on generating these matrices. Here we will illustrate the fifth condition, half of .

You should always get an empty list ([]) as the result of the second compute cell, no matter which random (unimodular) matrix you produce prior. Note that the list of vectors created using .columns() is exactly the list we want to feed to .linear_dependence().

In we proved which provided $n-r$ vectors that could be used with the span construction to build the entire null space of a matrix. As we have hinted in , and as we will see again going forward, linearly dependent sets carry redundant vectors with them when used in building a set as a span. Our aim now is to show that the vectors provided by form a linearly independent set, so in one sense they are as efficient as possible a way to describe the null space. Notice that the vectors $\vect{z}_j$, $1\leq j\leq n-r$ first appear in the vector form of solutions to arbitrary linear systems (). The exact same vectors appear again in the span construction in the conclusion of . Since this second theorem specializes to homogeneous systems the only real difference is that the vector $\vect{c}$ in is the zero vector for a homogeneous system. Finally, will now show that these same vectors are a linearly independent set. We will set the stage for the proof of this theorem with a moderately large example. Study the example carefully, as it will make it easier to understand the proof.

The proof of is really quite straightforward, and relies on the pattern of zeros and ones that arise in the vectors $\vect{z}_i$, $1\leq i\leq n-r$ in the entries that arise with the locations of the non-pivot columns. Play along with as you study the proof. Also, take a look at , and , especially at the conclusion of Step 2 (temporarily ignore the construction of the constant vector, $\vect{c}$). This proof is also a good first example of how to prove a conclusion that states a set is linearly independent.

Condensed — the full section is in Beezer, A First Course in Linear Algebra.