Change of basis

We have seen in that a linear transformation can be represented by a matrix, once we pick bases for the domain and codomain. How does the matrix representation change if we choose different bases? Which bases lead to especially nice representations? From the infinite possibilities, what is the best possible representation? This section will begin to answer these questions. But first we need to define eigenvalues for linear transformations and the change-of-basis matrix.

1. The change-of-basis matrix is a matrix representation of which linear transformation?
2. B&=\set{\colvector{2\\3},\,\colvector{-1\\2}}& C&=\set{\colvector{1\\0},\,\colvector{1\\1}}
3. surprise,

We now define the notion of an eigenvalue and eigenvector of a linear transformation. It should not be too surprising, especially if you remind yourself of the close relationship between matrices and linear transformations.

We will see shortly the best method for computing the eigenvalues and eigenvectors of a linear transformation, but for now, here are some examples to verify that such things really do exist.

Here is another.

Of course, these examples are meant only to illustrate the definition of eigenvectors and eigenvalues for linear transformations, and therefore beg the question, How would I find eigenvectors? We will have an answer before we finish this section. We need one more construction first.

Condensed: the full section is in Beezer, A First Course in Linear Algebra.

Given a vector space, we know we can usually find many different bases for the vector space, some nice, some nasty. If we choose a single vector from this vector space, we can build many different representations of the vector by constructing the representations relative to different bases. How are these different representations related to each other? A change-of-basis matrix answers this question.

Notice that this definition is primarily about a single vector space ($V$) and two bases of $V$ ($B$, $C$). The linear transformation ($I_V$) is necessary but not critical. As you might expect, this matrix has something to do with changing bases. Here is the theorem that gives the matrix its name (not the other way around).

So the change-of-basis matrix can be used with matrix multiplication to convert a vector representation of a vector ($\vect{v}$) relative to one basis ($\vectrep{B}{\vect{v}}$) to a representation of the same vector relative to a second basis ($\vectrep{C}{\vect{v}}$).

Condensed: the full section is in Beezer, A First Course in Linear Algebra.

Here is the main theorem of this section. It looks a bit involved at first glance, but the proof should make you realize it is not all that complicated. In any event, we are more interested in a special case.

We will be most interested in a special case of this theorem (), but here is an example that illustrates the full generality of .

Here is a special case of the previous theorem, where we choose $U$ and $V$ to be the same vector space, so the matrix representations and the change-of-basis matrices are all square of the same size.

This is the third surprise of this chapter. considers the special case where a linear transformation has the same vector space for the domain and codomain ($V$). We build a matrix representation of $T$ using the basis $B$ simultaneously for both the domain and codomain ($\matrixrep{T}{B}{B}$), and then we build a second matrix representation of $T$, now using the basis $C$ for both the domain and codomain ($\matrixrep{T}{C}{C}$). Then these two representations are related via a similarity transformation () using a change-of-basis matrix ($\cbm{B}{C}$)!

Condensed: the full section is in Beezer, A First Course in Linear Algebra.

tells us that the eigenvalues of a linear transformation are the eigenvalues of any representation, no matter what the choice of the basis $B$ might be. So we could now unambiguously define items such as the characteristic polynomial of a linear transformation, which we would define as the characteristic polynomial of any matrix representation. We will say that again eigenvalues, eigenvectors, and characteristic polynomials are intrinsic properties of a linear transformation, independent of the choice of a basis used to construct a matrix representation.

As a practical matter, how does one compute the eigenvalues and eigenvectors of a linear transformation of the form $\ltdefn{T}{V}{V}$? Choose a nice basis $B$ for $V$, one where the vector representations of the values of the linear transformations necessary for the matrix representation are easy to compute. Construct the matrix representation relative to this basis, and find the eigenvalues and eigenvectors of this matrix using the techniques of . The resulting eigenvalues of the matrix are precisely the eigenvalues of the linear transformation. The eigenvectors of the matrix are column vectors that need to be converted to vectors in $V$ through application of $\ltinverse{\vectrepname{B}}$ (this is part of the content of ).

Now consider the case where the matrix representation of a linear transformation is diagonalizable. The $n$ linearly independent eigenvectors that must exist for the matrix () can be converted (via $\ltinverse{\vectrepname{B}}$) into eigenvectors of the linear transformation. A matrix representation of the linear transformation relative to a basis of eigenvectors will be a diagonal matrix an especially nice representation! Though we did not know it at the time, the diagonalizations of were really about finding especially pleasing matrix representations of linear transformations.

Here are some examples.

Another example, this time a bit larger and with complex eigenvalues.

We finally have enough theorems to understand how Sage creates and manages linear transformations. With a choice of bases for the domain and codomain, a linear transformation can be represented by a matrix. Every interesting property of the linear transformation can be computed from the matrix representation, and we can convert between representations (of vectors and linear transformations) with change-of-basis matrices, similarity and matrix multiplication.

So we can understand the theory of linear algebra better by experimenting with the assistance of Sage, and the theory of linear algebra helps us understand how Sage is designed and functions. A virtuous cycle, if there ever was one. Keep it going.

Condensed: the full section is in Beezer, A First Course in Linear Algebra.

In mathematics, an ordered basis of a vector space of finite dimension n allows representing uniquely any element of the vector space by a coordinate vector, which is a finite sequence of n scalars called coordinates. If two different bases are considered, the coordinate vector that represents a vector v on one basis is, in general, different from the coordinate vector that represents v on the other basis. A change of basis consists of converting every assertion expressed in terms of coordinates relative to one basis into an assertion expressed in terms of coordinates relative to the other basis.

Such a conversion results from the change-of-basis formula, which expresses the coordinates relative to one basis in terms of the coordinates relative to the other basis. Using matrices, this formula can be written

\(\mathbf{x}_\mathrm{old} = A ~ \mathbf{x}_\mathrm{new},\)

where \(\mathbf{x}_\mathrm{old}\) and \(\mathbf{x}_\mathrm{new}\) are the column vectors of the coordinates of the same vector on the "old" (initially defined) and "new" (other) bases. \(A\) is the change-of-basis matrix (also called transition matrix), which is the matrix whose columns are the coordinates of the "new" basis vectors on the "old" basis.

A change of basis is sometimes called a change of coordinates, although it excludes many coordinate transformations. For applications in physics and specially in mechanics, a change of basis often involves the transformation of an orthonormal basis, understood as a rotation in physical space, thus excluding translations.

This article deals mainly with finite-dimensional vector spaces. However, many of the presented principles are also valid for infinite-dimensional vector spaces.

Let \(B_\mathrm{old} = (v_1, \ldots, v_n)\) be a basis of a finite-dimensional vector space V over a field F.

For j = 1, ..., n, one can define a vector w_j by its coordinates \(a_{i,j}\) over \(B_\mathrm{old} \colon\)

\(w_j = \sum_{i=1}^n a_{i,j} v_i.\)

Let

\(A = \left( a_{i,j} \right)_{i,j}\)

be the matrix whose j-th column is formed by the coordinates of w_j. (Here and in what follows, the index i refers always to the rows of A and the \(v_i,\) while the index j refers always to the columns of A and the \(w_j;\) such a convention is useful for avoiding errors in explicit computations.)

Setting \(B_\mathrm{new} = (w_1, \ldots, w_n),\) one has that \(B_\mathrm{new}\) is a basis of V if and only if the matrix A is invertible, or equivalently, if it has a nonzero determinant. In this case, A is said to be the change-of-basis matrix from the basis \(B_\mathrm{old}\) to the basis \(B_\mathrm{new}.\)

Given a vector \(u \in V,\) let \((x_1, \ldots, x_n)\) be the coordinates of \(u\) over \(B_\mathrm{old},\) and \((y_1, \ldots, y_n)\) its coordinates over \(B_\mathrm{new};\) that is

\(u = \sum_{i=1}^n x_i v_i = \sum_{j=1}^n y_j w_j.\)

(One could take the same summation index for the two sums, but choosing systematically the indexes i for the old basis and j for the new one makes clearer the formulas that follows, and helps avoiding errors in proofs and explicit computations.)

\(x_i = \sum_{j=1}^n a_{i,j} y_j \qquad \text{for } i=1, \ldots, n.\)

\(\mathbf{x} = A ~ \mathbf{y},\)

\(\begin{aligned} u &= \sum_{j=1}^n y_j w_j\\ &= \sum_{j=1}^n \left( y_j \sum_{i=1}^n a_{i,j} v_i \right)\\ &= \sum_{i=1}^n \left( \sum_{j=1}^n a_{i,j} y_j \right) v_i. \end{aligned}\)

Condensed: the full section is in Wikipedia.

Consider the Euclidean vector space \(\R^2\) and its standard basis, consisting of the vectors \(v_1\) and \(v_2\) with column vectors

\(\mathbf{v_1} = \begin{bmatrix} 1\\ 0 \end{bmatrix} \quad\) and \(\quad \mathbf{v_2} = \begin{bmatrix} 0\\ 1 \end{bmatrix}.\)

Rotating them by an angle of \(t\) gives a "new" basis, formed by the vectors \(w_1\) and \(w_2\) with column vectors

\(\mathbf{w_1} = \begin{bmatrix} \cos t\\ \sin t \end{bmatrix} \quad\) and \(\quad \mathbf{w_2} = \begin{bmatrix} -\sin t\\ \cos t \end{bmatrix}.\)

So, the change-of-basis matrix is

\(\begin{bmatrix} \cos t & -\sin t\\ \sin t & \cos t \end{bmatrix}.\)

The change-of-basis formula asserts that, for a vector with "old" coordinates \((x_1, x_2)\) and "new" coordinates \((y_1, y_2),\) one has

\(\begin{bmatrix} x_1\\ x_2 \end{bmatrix} = \begin{bmatrix} \cos t & -\sin t\\ \sin t & \cos t \end{bmatrix} ~ \begin{bmatrix} y_1\\ y_2 \end{bmatrix}.\)

That is,

\(\begin{cases} x_1 = y_1 \cos t - y_2 \sin t\\ x_2 = y_1 \sin t + y_2 \cos t. \end{cases}\)

This may be shown by writing

\(\begin{aligned} y_1 w_1 + y_2 w_2 &= y_1 \Big( (\cos t) ~ v_1 + (\sin t) ~ v_2 \Big) + y_2 \Big( \! -(\sin t) ~ v_1 + (\cos t) ~ v_2 \Big)\\ &= \big( y_1 \cos t - y_2 \sin t \big) v_1 + \big( y_1 \sin t + y_2 \cos t \big) v_2\\ &= x_1 v_1 + x_2 v_2. \end{aligned}\)

Usually, a matrix represents a linear map, and the product of a matrix and a column vector represents the function application of the corresponding linear map to the vector whose coordinates form the column vector. The change-of-basis formula is a specific case of this general principle, although this is not immediately clear from its definition and proof.

When one says that a matrix represents a linear map, one refers implicitly to bases of implied vector spaces, and to the fact that the choice of a basis induces a linear isomorphism between a vector space and \(F^n\), where F is the ground field of scalars. When only one basis is considered for each vector space, it is convenient to leave this isomorphism implicit, and to work up to an isomorphism. As several bases of the same vector space are considered here, a more accurate wording is required.

Let F be a field, the set \(F^n\) of the n-tuples is an F-vector space whose addition and scalar multiplication are defined component-wise. Its standard basis is the basis that has as its i-th element the tuple with all components equal to 0 except the i-th one, equal to 1.

A basis \(B = (v_1, \ldots, v_n)\) of an F-vector space V defines a linear isomorphism \(\varphi \colon F^n \to V\) by

\(\varphi(x_1, \ldots, x_n) = \sum_{i=1}^n x_i v_i.\)

Conversely, such a linear isomorphism defines a basis, which is the image by \(\varphi\) of the standard basis of \(F^n.\)

Let \(B_\mathrm{old} = (v_1, \ldots, v_n)\) be the "old" basis of a change of basis, and \(\varphi_\mathrm{old}\) the associated isomorphism. Given a change-of-basis (invertible) matrix A, one could consider it the matrix of an automorphism (a bijective endomorphism) \(\psi_A\) of \(F^n.\) Finally, define

\(\varphi_\mathrm{new} = \varphi_\mathrm{old} \circ \psi_A\)

(where \(\circ\) denotes function composition), and

\(B_\mathrm{new} = \varphi_\mathrm{new} \left( \varphi_\mathrm{old}^{-1}(B_\mathrm{old}) \right).\)

\(\varphi_\mathrm{old}^{-1} = \psi_A \circ \varphi_\mathrm{new}^{-1}.\)

\(\varphi_\mathrm{old}^{-1}(v) = \psi_A \left( \varphi_\mathrm{new}^{-1}(v) \right),\)

Condensed: the full section is in Wikipedia.

A function that has a vector space as its domain is commonly specified as a multivariate function whose variables are the coordinates on some basis of the vector on which the function is applied.

When the basis is changed, the expression of the function is changed. This change can be computed by substituting the "old" coordinates for their expressions in terms of the "new" coordinates. More precisely, if f(x) is the expression of the function in terms of the "old" coordinates, and if x = Ay is the change-of-basis formula, then f(Ay) is the expression of the same function in terms of the "new" coordinates.

The fact that the change-of-basis formula expresses the "old" coordinates in terms of the "new" ones may seem unnatural, but appears as useful, because no matrix inversion is needed here.

As the change-of-basis formula involves only linear functions, many function properties are kept by a change of basis. This allows defining these properties as properties of functions of a variable vector that are not related to any specific basis. So, a function whose domain is a vector space or a subset of it is

• a linear function,
• a polynomial function,
• a continuous function,
• a differentiable function,
• a smooth function,
• an analytic function,

if the multivariate function that represents it on some basis, and thus on every basis, has the same property.

This is specially useful in the theory of manifolds, as this allows extending the concepts of continuous, differentiable, smooth, and analytic functions to functions that are defined on a manifold.

Consider a linear map L: V → W from a vector space V of dimension n to a vector space W of dimension m. It is represented on "old" bases of V and W by an m×n matrix M. A change of basis is defined by an n×n change-of-basis matrix P for V, and an m×m change-of-basis matrix Q for W.

On the "new" bases, the matrix representation of L is

\(M' = Q^{-1}MP.\)

This is a straightforward consequence of the change-of-basis formula.

Endomorphisms are linear maps from a vector space V to itself. For a change of basis, the formula of the preceding section applies, with the same change-of-basis matrix on both sides of the formula. That is, if M is the square matrix of an endomorphism of V on an "old" basis, and P is a change-of-basis matrix, then the matrix of the endomorphism on the "new" basis is

\(M' = P^{-1}MP.\)

As every invertible matrix can be used as a change-of-basis matrix, this implies that two matrices are similar if and only if they represent the same endomorphism on two different bases.

A bilinear form on a vector space V over a field F is a function \(\varPhi \colon V \! \times V \to F\) which is linear in both arguments. That is, \(\varPhi\) is bilinear if the maps \(v \mapsto \varPhi(v, w)\) and \(v \mapsto \varPhi(w, v)\) are linear for every fixed \(w \in V.\)

The matrix \(\mathbf{\Phi}\) of a bilinear form \(\varPhi\) on a basis \((v_1, \ldots, v_n)\) (the "old" basis) is the matrix whose entry of the i-th row and j-th column is \(\varPhi(v_i, v_j)\). It follows that for two vectors v and w with coordinate column vectors v and w, one has

\(\varPhi(v, w) = \mathbf{v}^{\mathsf T} \mathbf{\Phi} \mathbf{w},\)

where \(\mathbf{v}^{\mathsf T}\) denotes the transpose of the column vector v.

For a change of basis with matrix P, a straightforward computation shows that the matrix of the bilinear form on the "new" basis is

\(\mathbf{\Phi'} = P^{\mathsf T} \mathbf{\Phi} P.\)

A symmetric bilinear form is a bilinear form S such that \(S(v,w) = S(w,v)\) for every v and w in V. It follows that the matrix of S on any basis is symmetric. This implies that the property of being a symmetric matrix must be kept by the above change-of-basis formula. One can also check this by noting that the transpose of a matrix product is the product of the transposes computed in the reverse order. Thus,

\((P^{\mathsf T} \mathbf{S} P)^{\mathsf T} = P^{\mathsf T} \mathbf{S}^{\mathsf T} P;\)

finally, since the matrix S is symmetric,

\((P^{\mathsf T} \mathbf{S} P)^{\mathsf T} = P^{\mathsf T} \mathbf{S} P.\)

If the characteristic of the ground field F is not two, then for every symmetric bilinear form, there is a basis for which the matrix is diagonal. Moreover, the resulting nonzero entries on the diagonal are defined up to the multiplication by a square. So, if the ground field is the field \(\R\) of the real numbers, these nonzero entries can be chosen to be either 1 or –1. Sylvester's law of inertia is a theorem that asserts that the numbers of 1 and of –1 depend only on the bilinear form, and not on the change of basis.

Condensed: the full section is in Wikipedia.