maths.free › Linear Algebra › 2. Vectors › Linear Dependence and Spans

Linear Dependence and Spans

In any linearly dependent set there is always one vector that can be written as a linear combination of the others. This is the substance of the upcoming . Perhaps this will explain the use of the word dependent. In a linearly dependent set, at least one vector depends on the others (via a linear combination).

Indeed, because is an equivalence () some authors use this condition as a definition () of linear dependence. Then linear independence is defined as the logical opposite of linear dependence. Of course, we have chosen to take as our definition, and then follow with as a theorem.

S=\set{\colvector{1\\10\\100\\1000},\,\colvector{1\\1\\1\\1},\,\colvector{5\\23\\203\\2003}}
dependent
Suppose that $Y=\spn{P}=\spn{Q}$, where $P$ is a linearly dependent set and $Q$ is linearly independent. Would you rather use $P$ or $Q$ to describe $Y$? Why?

Linearly Dependent Sets and Spans

If we use a linearly dependent set to construct a span, then we can always create the same infinite set with a starting set that is one vector smaller in size. We will illustrate this behavior in . However, this will not be possible if we build a span from a linearly independent set. So in a certain sense, using a linearly independent set to formulate a span is the best possible way there are not any extra vectors being used to build up all the necessary linear combinations. OK, here is the theorem, and then the example.

Theorem: Dependency in Linearly Dependent Sets

Suppose that $S=\set{\vectorlist{u}{n}}$ is a set of vectors. Then $S$ is a linearly dependent set if and only if there is an index $t$, $1\leq t\leq n$ such that $\vect{u_t}$ is a linear combination of the vectors $\vect{u}_1,\,\vect{u}_2,\,\vect{u}_3,\,\ldots,\,\vect{u}_{t-1},\,\vect{u}_{t+1},\,\ldots,\,\vect{u}_n$.

Proof

Suppose that $S$ is linearly dependent, so there exists a nontrivial relation of linear dependence by . That is, there are scalars, $\alpha_i$, $1\leq i\leq n$, which are not all zero, such that \lincombo{\alpha}{u}{n}=\zerovector. Since the $\alpha_i$ cannot all be zero, choose one, say $\alpha_t$, that is nonzero. Then, \vect{u}_t &=\frac{-1}{\alpha_t}\left(-\alpha_t\vect{u}_t\right)&&\\ &= \frac{-1}{\alpha_t}\left( \alpha_1\vect{u}_1+ \cdots+ \alpha_{t-1}\vect{u}_{t-1}+ \alpha_{t+1}\vect{u}_{t+1}+ \cdots+ \alpha_n\vect{u}_n \right)&&\\ &= \frac{-\alpha_1}{\alpha_t}\vect{u}_1+ \cdots+ \frac{-\alpha_{t-1}}{\alpha_t}\vect{u}_{t-1}+ \frac{-\alpha_{t+1}}{\alpha_t}\vect{u}_{t+1}+ \cdots+ \frac{-\alpha_n}{\alpha_t}\vect{u}_n &&

Since the values of $\frac{\alpha_i}{\alpha_t}$ are again scalars, we have expressed $\vect{u}_t$ as a linear combination of the other elements of $S$.

Assume that the vector $\vect{u}_t$ is a linear combination of the other vectors in $S$. Write this linear combination, denoting the relevant scalars as $\beta_1$, $\beta_2$, , $\beta_{t-1}$, $\beta_{t+1}$, , $\beta_n$, as \vect{u_t} &= \beta_1\vect{u}_1+ \beta_2\vect{u}_2+ \cdots+ \beta_{t-1}\vect{u}_{t-1}+ \beta_{t+1}\vect{u}_{t+1}+ \cdots+ \beta_n\vect{u}_n

Then we have \beta_1\vect{u}_1 &+\cdots+ \beta_{t-1}\vect{u}_{t-1}+ (-1)\vect{u}_t+ \beta_{t+1}\vect{u}_{t+1}+ \cdots+ \beta_n\vect{u}_n\\ &=\vect{u}_t+(-1)\vect{u}_t&&\\ &=\left(1+\left(-1\right)\right)\vect{u}_t&&\\ &=0\vect{u}_t&&\\ &=\zerovector&&

So the scalars $\beta_1,\,\beta_2,\,\beta_3,\,\ldots,\,\beta_{t-1},\,\beta_t=-1,\beta_{t+1},\,\,\ldots,\,\beta_n$ provide a nontrivial linear combination of the vectors in $S$, thus establishing that $S$ is a linearly dependent set ().

Condensed — the full section is in Beezer, A First Course in Linear Algebra.

Casting Out Vectors

In we used four vectors to create a span. With a relation of linear dependence in hand, we were able to toss out one of these four vectors and create the same span from a subset of just three vectors from the original set of four. We did have to take some care as to just which vector we tossed out. In the next example, we will be more methodical about just how we choose to eliminate vectors from a linearly dependent set while preserving a span.

Casting Out Vectors

We will redo , though somewhat tersely, just producing the justification for each time we toss a vector (a specific relation of linear dependence), and then verifying that the resulting spans, each with one fewer vector, still produce the original span. We also introduce the .remove() method for lists. Ready? Here we go.

Notice that S begins with all seven original vectors, and slowly gets whittled down to just the list [v1, v3, v4]. If you experiment with the above commands, be sure to return to the start and work your way through in order, or the results will not be right.

As a bonus, notice that the set of relations of linear dependence provided by Sage, D, is itself a linearly independent set (but within a very different vector space). Is that too weird?

Now, can you answer the extra credit question from using Sage?

deserves your careful attention, since this important example motivates the following very fundamental theorem.

In , we tossed-out vectors one at a time. But in each instance, we rewrote the offending vector as a linear combination of those vectors with the column indices of the pivot columns of the reduced row-echelon form of the matrix of columns. In the proof of , we accomplish this reduction in one big step. In we arrived at a linearly independent set at exactly the same moment that we ran out of free variables to exploit. This was not a coincidence, it is the substance of our conclusion of linear independence in .

Here is a straightforward application of .

Reducing a Span

allows us to construct a reduced spanning set for a span. As with the theorem, employing Sage we begin by constructing a matrix with the vectors of the spanning set as columns. Here is a do-over of , illustrating the use of in Sage.

Notice how we compute T with the single line that mirrors the construction of the set $T=\set{\vect{v}_{d_1},\,\vect{v}_{d_2},\,\vect{v}_{d_3},\,\ldots\,\vect{v}_{d_r}}$ in the statement of . Again, the row-reducing is hidden in the use of the .pivot() matrix method, which necessarily must compute the reduced row-echelon form. The final two compute cells verify both conclusions of the theorem.

Condensed — the full section is in Beezer, A First Course in Linear Algebra.

Symbols used here

imaginary unit
i² = −1.

\leq,\ \geq

less/greater than or equal
Inequalities that allow equality; < and > exclude it.

\mathbf{v},\ \vec{v}

vector
A quantity with magnitude and direction; a column of numbers.

A = \begin{pmatrix} a & b \\ c & d \end{pmatrix}

matrix
A rectangular array of numbers; a linear map.

\det A,\ |A|

determinant
Scaling factor of area/volume under A; zero means singular.

A^{-1},\ A^{T}

inverse, transpose
The matrix that undoes A; A with rows and columns swapped.

\lambda

lambda (eigenvalue)
The factor by which an eigenvector is stretched: Av = λv.

\mathbf{u} \cdot \mathbf{v},\ \|\mathbf{v}\|

dot product, norm
Σ u_i v_i; the length of v, √(v·v).

Questions people ask

What does a determinant mean geometrically?

It is the factor by which the matrix scales area (2×2) or volume (3×3), with a negative sign if orientation flips. Zero means the matrix flattens space and cannot be undone.

What is an eigenvector?

A direction the matrix does not turn — it only stretches it by the eigenvalue. Along eigenvectors a complicated matrix acts like multiplication by a number.

Why is matrix multiplication not commutative?

Because a matrix is a transformation and AB means "do B, then A". Rotating then reflecting is not the same as reflecting then rotating.

Özüňi synla

Parts of this page are adapted from Beezer, A First Course in Linear Algebra (GFDL 1.2). Condensed and re-explained here; errors are ours.

_Ýaşa Linear Algebra

Determinants Matrix inverse Eigenvalues and eigenvectors Matrix multiplication Row reduction Vector spaces, span and linear independence Orthogonality, projections and least squares Diagonalisation and matrix powers Linear transformations and change of basis