Trace (linear algebra)

In linear algebra, the trace of a square matrix A, denoted tr(A), is defined as a sum of the elements on its main diagonal, \(a_{11} + a_{22} + \dots + a_{nn}\). It is only defined for a square matrix (n × n).

It can be shown that the trace of a matrix is equal to the sum of its eigenvalues (counted with algebraic multiplicities), see below. Also, tr(AB) = tr(BA) for any matrices A and B of the same size. Thus, similar matrices have the same trace. As a consequence, one can define the trace of a linear operator mapping a finite-dimensional vector space into itself, since all matrices describing such an operator with respect to a basis are similar.

The trace is related to the derivative of the determinant (see Jacobi's formula).

The trace of an n × n square matrix A is defined as \[\operatorname{tr}(\mathbf{A}) = \sum_{i=1}^n a_{ii} = a_{11} + a_{22} + \dots + a_{nn}\] where a_ii denotes the entry on the i th row and i th column of A. The entries of A can be real numbers, complex numbers, or more generally elements of a field F. The trace is not defined for non-square matrices.

Let A be a matrix, with \[\mathbf{A} = \begin{pmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{pmatrix} = \begin{pmatrix} 1 & 0 & 3 \\ 11 & 5 & 2 \\ 6 & 12 & -5 \end{pmatrix}\]

Then \[\operatorname{tr}(\mathbf{A}) = \sum_{i=1}^{3} a_{ii} = a_{11} + a_{22} + a_{33} = 1 + 5 + (-5) = 1.\]

The trace is a linear mapping. That is, \[\begin{aligned} \operatorname{tr}(\mathbf{A} + \mathbf{B}) &= \operatorname{tr}(\mathbf{A}) + \operatorname{tr}(\mathbf{B}) \\ \operatorname{tr}(c\mathbf{A}) &= c \operatorname{tr}(\mathbf{A}) \end{aligned}\] for all square matrices A and B, and all scalars c.

A matrix and its transpose have the same trace: \[\operatorname{tr}(\mathbf{A}) = \operatorname{tr}\left(\mathbf{A}^\mathsf{T}\right).\]

This follows immediately from the fact that transposing a square matrix does not affect elements along the main diagonal.

The trace of a square matrix which is the product of two matrices can be rewritten as the sum of entry-wise products of their elements, i.e. as the sum of all elements of their Hadamard product. Phrased directly, if A and B are two m × n matrices, then: \[\operatorname{tr}\left(\mathbf{A}^\mathsf{T}\mathbf{B}\right) = \operatorname{tr}\left(\mathbf{A}\mathbf{B}^\mathsf{T}\right) = \operatorname{tr}\left(\mathbf{B}^\mathsf{T}\mathbf{A}\right) = \operatorname{tr}\left(\mathbf{B}\mathbf{A}^\mathsf{T}\right) = \sum_{i=1}^m \sum_{j=1}^n a_{ij}b_{ij} \; .\]

If one views any real m × n matrix as a vector of length mn (an operation called vectorization) then the above operation on A and B coincides with the standard dot product. According to the above expression, tr(AA) is a sum of squares and hence is nonnegative, equal to zero if and only if A is zero. Furthermore, as noted in the above formula, tr(AB) = tr(BA). These demonstrate the positive-definiteness and symmetry required of an inner product; it is common to call tr(AB) the Frobenius inner product of A and B. This is a natural inner product on the vector space of all real matrices of fixed dimensions. The norm derived from this inner product is called the Frobenius norm, and it satisfies a submultiplicative property, as can be proven with the Cauchy-Schwarz inequality: \[0 \leq \left[\operatorname{tr}(\mathbf{A} \mathbf{B})\right]^2 \leq \operatorname{tr}\left(\mathbf{A}^\mathsf{T} \mathbf{A}\right) \operatorname{tr}\left(\mathbf{B}^\mathsf{T} \mathbf{B}\right) ,\] if A and B are real matrices such that A B is a square matrix. The Frobenius inner product and norm arise frequently in matrix calculus and statistics.

The Frobenius inner product may be extended to a hermitian inner product on the complex vector space of all complex matrices of a fixed size, by replacing B by its complex conjugate.

The symmetry of the Frobenius inner product may be phrased more directly as follows: the matrices in the trace of a product can be switched without changing the result. If A and B are m × n and n × m real or complex matrices, respectively, then

\(\operatorname{tr}(\mathbf{A}\mathbf{B}) = \operatorname{tr}(\mathbf{B}\mathbf{A})\)

This is notable both for the fact that AB does not usually equal BA, and also since the trace of either does not usually equal tr(A)tr(B). The similarity-invariance of the trace, meaning that tr(A) = tr(PAP) for any square matrix A and any invertible matrix P of the same dimensions, is a fundamental consequence. This is proved by \[\operatorname{tr}\left(\mathbf{P}^{-1}(\mathbf{A}\mathbf{P})\right) = \operatorname{tr}\left((\mathbf{A} \mathbf{P})\mathbf{P}^{-1}\right) = \operatorname{tr}(\mathbf{A}).\] Similarity invariance is the crucial property of the trace in order to discuss traces of linear transformations as below.

Additionally, for real column vectors \(\mathbf{a}\in\mathbb{R}^n\) and \(\mathbf{b}\in\mathbb{R}^n\), the trace of the outer product is equivalent to the inner product:

\(\operatorname{tr}\left(\mathbf{b}\mathbf{a}^\textsf{T}\right) = \mathbf{a}^\textsf{T}\mathbf{b}\)

More generally, the trace is invariant under circular shifts, that is,

\(\operatorname{tr}(\mathbf{A}\mathbf{B}\mathbf{C}) = \operatorname{tr}(\mathbf{B}\mathbf{C}\mathbf{A}) = \operatorname{tr}(\mathbf{C}\mathbf{A}\mathbf{B}).\)

This is known as the cyclic property.

Arbitrary permutations are not allowed: in general, \[\operatorname{tr}(\mathbf{A}\mathbf{B}\mathbf{C}) \ne \operatorname{tr}(\mathbf{A}\mathbf{C}\mathbf{B}) .\]

However, if products of three symmetric matrices are considered, any permutation is allowed, since: \[\operatorname{tr}(\mathbf{A}\mathbf{B}\mathbf{C}) = \operatorname{tr}\left(\left(\mathbf{A}\mathbf{B}\mathbf{C}\right)^{\mathsf T}\right) = \operatorname{tr}(\mathbf{C}\mathbf{B}\mathbf{A}) = \operatorname{tr}(\mathbf{A}\mathbf{C}\mathbf{B}),\] where the first equality is because the traces of a matrix and its transpose are equal. Note that this is not true in general for more than three factors.

The trace of the Kronecker product of two matrices is the product of their traces: \[\operatorname{tr}(\mathbf{A} \otimes \mathbf{B}) = \operatorname{tr}(\mathbf{A})\operatorname{tr}(\mathbf{B}).\]

The following three properties: \[\begin{aligned} \operatorname{tr}(\mathbf{A} + \mathbf{B}) &= \operatorname{tr}(\mathbf{A}) + \operatorname{tr}(\mathbf{B}), \\ \operatorname{tr}(c\mathbf{A}) &= c \operatorname{tr}(\mathbf{A}), \\ \operatorname{tr}(\mathbf{A}\mathbf{B}) &= \operatorname{tr}(\mathbf{B}\mathbf{A}), \end{aligned}\] characterize the trace up to a scalar multiple; in other words: If \(f\) is a linear functional on the space of square matrices that satisfies \(f(xy) = f(yx),\) then \(f\) and \(\operatorname{tr}\) are proportional.

For \(n\times n\) matrices, imposing the normalization \(f(\mathbf{I}) = n\) makes \(f\) equal to the trace.

Given any n × n matrix A, there is

\(\operatorname{tr}(\mathbf{A}) = \sum_{i=1}^n \lambda_i\)

where λ₁, ..., λ_n are the eigenvalues of A counted with algebraic multiplicity. This holds true even if A is a real matrix and some (or all) of the eigenvalues are complex numbers, or more generally over any field with eigenvalues taken in an algebraic closure. The identity follows from the fact that A is always similar to its Jordan form, an upper triangular matrix having λ₁, ..., λ_n on the main diagonal, together with the similarity-invariance of the trace discussed above. In contrast, the determinant of A is the product of its eigenvalues; that is, \[\det(\mathbf{A}) = \prod_i \lambda_i.\]

When both A and B are n × n matrices, the trace of the (ring-theoretic) commutator of A and B vanishes: tr([A, B]) = 0, because tr(AB) = tr(BA) and tr is linear. One can state this as "the trace is a map of Lie algebras gl_n → k from operators to scalars", as the commutator of scalars is trivial (it is an Abelian Lie algebra). In particular, using similarity invariance, it follows that the identity matrix is never similar to the commutator of any pair of matrices.

Conversely, any square matrix with zero trace is a linear combination of the commutators of pairs of matrices. Moreover, any square matrix with zero trace is unitarily equivalent to a square matrix with diagonal consisting of all zeros.

• The trace of the n × n identity matrix is the dimension of the space, namely n. \[\operatorname{tr}\left(\mathbf{I}_n\right) = n\] This leads to generalizations of dimension using trace.
• The trace of a Hermitian matrix is real, because the elements on the diagonal are real.
• The trace of a permutation matrix is the number of fixed points of the corresponding permutation, because the diagonal term a_ii is 1 if the ith point is fixed and 0 otherwise.
• The trace of an orthogonal projection matrix is the dimension of the target space. \[\begin{aligned} \mathbf{P}_\mathbf{X} &= \mathbf{X}\left(\mathbf{X}^\mathsf{T} \mathbf{X}\right)^{-1} \mathbf{X}^\mathsf{T} \\[3pt] \Longrightarrow \operatorname{tr}\left(\mathbf{P}_\mathbf{X}\right) &= \operatorname{tr}\left(\mathbf{X}^\mathsf{T}\mathbf{X}\left(\mathbf{X}^\mathsf{T} \mathbf{X}\right)^{-1}\right)=\operatorname{rank}(\mathbf{X}). \end{aligned}\]
• More generally, the trace of any projection, or idempotent matrix, i.e. one with A = A, equals its own rank, for instance since A only has the eigenvalues 1 and 0, with 1 having multiplicity \(\operatorname{rank}(\mathbf A)\).
• The trace of a nilpotent matrix is zero.

  When the characteristic of the base field is zero, the converse also holds: if tr(A) = 0 for all k, then A is nilpotent.

  When the characteristic n > 0 is positive, the identity in n dimensions is a counterexample, as \(\operatorname{tr}\left(\mathbf{I}_n^k\right) = \operatorname{tr}\left(\mathbf{I}_n\right) = n \equiv 0\), but the identity is not nilpotent.

The trace of an \(n \times n\) matrix \(A\) is the coefficient of \(t^{n-1}\) in the characteristic polynomial, possibly changed of sign, according to the convention in the definition of the characteristic polynomial.

If a is a square matrix with small entries and I denotes the identity matrix, then we have approximately

\[\det(\mathbf{I}+\mathbf{a})\approx 1 + \operatorname{tr}(\mathbf{a}).\]

Precisely this means that the trace is the derivative of the determinant function at the identity matrix. Jacobi's formula

\[d\det(\mathbf{A}) = \operatorname{tr} \big(\operatorname{adj}(\mathbf{A})\cdot d\mathbf{A}\big)\]

is more general and describes the differential of the determinant at an arbitrary square matrix, in terms of the trace and the adjugate of the matrix.

From this (or from the connection between the trace and the eigenvalues), one can derive a relation between the trace function, the matrix exponential function, and the determinant:\[\det(\exp(\mathbf{A})) = \exp(\operatorname{tr}(\mathbf{A})).\]

A related characterization of the trace applies to linear vector fields. Given a matrix A, define a vector field F on R by F(x) = Ax. The components of this vector field are linear functions (given by the rows of A). Its divergence div F is a constant function, whose value is equal to tr(A).

Condensed: the full section is in Wikipedia.

In general, given some linear map f : V → V of finite rank (where V is a vector space), we can define the trace of this map by considering the trace of a matrix representation of f, that is, choosing a basis for V and describing f as a matrix relative to this basis, and taking the trace of this square matrix. The result will not depend on the basis chosen, since different bases will give rise to similar matrices, allowing for the possibility of a basis-independent definition for the trace of a linear map.

Such a definition can be given using the canonical isomorphism between the space of linear endomorphisms of V of finite rank and V ⊗ V*, where V* is the dual space of V. Let v be in V and let g be in V*. Then the trace of the decomposable element v ⊗ g is defined to be g(v); the trace of a general element is defined by linearity. The trace of a linear map f : V → V of finite rank can then be defined as the trace, in the above sense, of the element of V ⊗ V* corresponding to f under the above-mentioned canonical isomorphism. Using an explicit basis for V and the corresponding dual basis for V*, one can show that this gives the same definition of the trace as given above.