Positive-definite matrix

In mathematics, a symmetric matrix \(M\) with real entries is positive-definite if the real number \(\mathbf{x}^\mathsf{T} M \mathbf{x}\) is positive for every nonzero real column vector \(\mathbf{x},\) where \(\mathbf{x}^\mathsf{T}\) is the row vector transpose of \(\mathbf{x}.\) More generally, a Hermitian matrix (that is, a complex matrix equal to its conjugate transpose) is positive-definite if the real number \(\mathbf{z}^* M \mathbf{z}\) is positive for every nonzero complex column vector \(\mathbf{z},\) where \(\mathbf{z}^*\) denotes the conjugate transpose of \(\mathbf{z}.\)

Positive semi-definite matrices are defined similarly, except that the scalars \(\mathbf{x}^\mathsf{T} M \mathbf{x}\) and \(\mathbf{z}^* M \mathbf{z}\) are required to be positive or zero (that is, nonnegative). Negative-definite and negative semi-definite matrices are defined analogously. A matrix that is not positive semi-definite and not negative semi-definite is sometimes called indefinite.

Some authors use more general definitions of definiteness, permitting the matrices to be non-symmetric or non-Hermitian. The properties of these generalized definite matrices are explored in § Extension for non-Hermitian square matrices, below, but are not the main focus of this article.

In the following definitions, \(\mathbf{x}^\mathsf{T}\) is the transpose of \(\mathbf{x},\) \(\mathbf{z}^*\) is the conjugate transpose of \(\mathbf{z},\) and \(\mathbf{0}\) denotes the n dimensional zero-vector.

An \(n \times n\) symmetric real matrix \(M\) is said to be positive-definite if \(\mathbf{x}^\mathsf{T} M\mathbf{x} > 0\) for all non-zero \(\mathbf{x}\) in \(\mathbb{R}^n.\) Formally, \[M \text{ positive-definite} \quad \iff \quad \mathbf{x}^\mathsf{T} M\mathbf{x} > 0 \text{ for all } \mathbf{x} \in \R^n \setminus \{\mathbf{0}\}\]

An \(n \times n\) symmetric real matrix \(M\) is said to be positive-semidefinite or non-negative-definite if \(\mathbf{x}^\mathsf{T} M\mathbf{x} \geq 0\) for all \(\mathbf{x}\) in \(\mathbb{R}^n .\) Formally, \[M \text{ positive semi-definite} \quad \iff \quad \mathbf{x}^\mathsf{T} M\mathbf{x} \geq 0 \text{ for all } \mathbf{x} \in \mathbb{R}^n\]

An \(n \times n\) symmetric real matrix \(M\) is said to be negative-definite if \(\mathbf{x}^\mathsf{T} M\mathbf{x} < 0\) for all non-zero \(\mathbf{x}\) in \(\R^n.\) Formally, \[M \text{ negative-definite} \quad \iff \quad \mathbf{x}^\mathsf{T} M\mathbf{x} < 0 \text{ for all } \mathbf{x} \in \mathbb{R}^n \setminus \{\mathbf{0}\}\]

An \(n \times n\) symmetric real matrix \(M\) is said to be negative-semidefinite or non-positive-definite if \(\mathbf{x}^\mathsf{T} M\mathbf{x} \leq 0\) for all \(\mathbf{x}\) in \(\mathbb{R}^n .\) Formally, \[M \text{ negative semi-definite} \quad \iff \quad \mathbf{x}^\mathsf{T} M\mathbf{x} \leq 0 \text{ for all } \mathbf{x} \in \R^n\]

An \(n \times n\) symmetric real matrix which is neither positive semidefinite nor negative semidefinite is called indefinite.

The following definitions all involve the term \(\mathbf{z}^* M\mathbf{z}.\) Notice that this is always a real number for any Hermitian square matrix \(M.\)

An \(n \times n\) Hermitian complex matrix \(M\) is said to be positive-definite if \(\mathbf{z}^* M\mathbf{z} > 0\) for all non-zero \(\mathbf{z}\) in \(\mathbb{C}^n .\) Formally,

\(M \text{ positive-definite} \quad \iff \quad \mathbf{z}^* M\mathbf{z} > 0 \text{ for all } \mathbf{z} \in \mathbb{C}^n \setminus \{ \mathbf{0} \}\)

An \(n \times n\) Hermitian complex matrix \(M\) is said to be positive semi-definite or non-negative-definite if \(\mathbf{z}^* M\mathbf{z} \geq 0\) for all \(\mathbf{z}\) in \(\mathbb{C}^n .\) Formally,

\(M \text{ positive semi-definite} \quad \iff \quad \mathbf{z}^* M\mathbf{z} \geq 0 \text{ for all } \mathbf{z} \in \mathbb{C}^n\)

An \(n \times n\) Hermitian complex matrix \(M\) is said to be negative-definite if \(\mathbf{z}^* M\mathbf{z} < 0\) for all non-zero \(\mathbf{z}\) in \(\mathbb{C}^n .\) Formally,

\(M \text{ negative-definite} \quad \iff \quad \mathbf{z}^* M\mathbf{z} < 0 \text{ for all } \mathbf{z} \in \mathbb{C}^n \setminus \{\mathbf{0}\}\)

Condensed: the full section is in Wikipedia.

Since every real matrix is also a complex matrix, the definitions of "definiteness" for the two classes must agree.

For complex matrices, the most common definition says that \(M\) is positive-definite if and only if \(\mathbf{z}^* M\mathbf{z}\) is real and positive for every non-zero complex column vectors \(\mathbf{z} .\) This condition implies that \(M\) is Hermitian (i.e. its transpose is equal to its conjugate), since \(\mathbf{z}^* M\mathbf{z}\) being real, it equals its conjugate transpose \(\mathbf{z}^*M^*\mathbf{z}\) for every \(\mathbf{z},\) which implies \(M = M^* .\)

By this definition, a positive-definite real matrix \(M\) is Hermitian, hence symmetric; and \(\mathbf{z}^\mathsf{T} M\mathbf{z}\) is positive for all non-zero real column vectors \(\mathbf{z} .\) However the last condition alone is not sufficient for \(M\) to be positive-definite. For example, if \[M = \begin{bmatrix} 1 & 1 \\-1 & 1 \end{bmatrix},\]

then for any real vector \(\mathbf{z}\) with entries \(a\) and \(b\) we have \(\mathbf{z}^\mathsf{T} M\mathbf{z} = \left(a + b\right)a + \left(-a + b\right) b = a^2 + b^2,\) which is always positive if \(\mathbf{z}\) is not zero. However, if \(\mathbf{z}\) is the complex vector with entries 1 and ⁠\(i\)⁠, one gets

\[\mathbf{z}^* M\mathbf{z} = \begin{bmatrix} 1 & -i \end{bmatrix}M\begin{bmatrix} 1 \\i \end{bmatrix} = \begin{bmatrix} 1 + i & 1 - i \end{bmatrix}\begin{bmatrix} 1 \\i \end{bmatrix} = 2 + 2i .\]

which is not real. Therefore, \(M\) is not positive-definite.

On the other hand, for a symmetric real matrix \(M,\) the condition "\(\mathbf{z}^\mathsf{T} M\mathbf{z} > 0\) for all nonzero real vectors \(\mathbf{z}\)" does imply that \(M\) is positive-definite in the complex sense.

If a Hermitian matrix \(M\) is positive semi-definite, one sometimes writes \(M \succeq 0\) and if \(M\) is positive-definite one writes \(M \succ 0.\) To denote that \(M\) is negative semi-definite one writes \(M \preceq 0\) and to denote that \(M\) is negative-definite one writes \(M \prec 0.\)

The notation comes from functional analysis where positive semidefinite matrices define positive operators. If two matrices \(A\) and \(B\) satisfy \(B - A \succeq 0,\) we can define a non-strict partial order \(B \succeq A\) that is reflexive, antisymmetric, and transitive; it is not a total order, however, as \(B - A,\) in general, may be indefinite. Also note that \(\succeq\) and \(\succ\) do not satisfy the usual correspondence of non-strict and strict partial order relations, because \(B \succeq A \land B \neq A\) does not imply \(B \succ A\).

A common alternative notation is \(M \geq 0,\) \(M > 0,\) \(M \leq 0,\) and \(M < 0\) for positive semi-definite and positive-definite, negative semi-definite and negative-definite matrices, respectively. This may be confusing, as sometimes nonnegative matrices (respectively, nonpositive matrices) are also denoted in this way.

It follows from the above definitions that a Hermitian matrix is positive-definite if and only if it is the matrix of a positive-definite quadratic form or Hermitian form. In other words, a Hermitian matrix is positive-definite if and only if it defines an inner product.

Positive-definite and positive-semidefinite matrices can be characterized in many ways, which may explain the importance of the concept in various parts of mathematics. A Hermitian matrix M is positive-definite if and only if it satisfies any of the following equivalent conditions.

• \(M\) is congruent with a diagonal matrix with positive real entries.
• \(M\) is Hermitian, and all its eigenvalues are real and positive.
• \(M\) is Hermitian, and all its leading principal minors are positive.
• There exists an invertible matrix \(B\) with conjugate transpose \(B^*\) such that \(M = B^* B.\)

A matrix is positive semi-definite if it satisfies similar equivalent conditions where "positive" is replaced by "nonnegative", "invertible matrix" is replaced by "matrix", and the word "leading" is removed.

Positive-definite and positive-semidefinite real matrices are at the basis of convex optimization, since, given a function of several real variables that is twice differentiable, then if its Hessian matrix (matrix of its second partial derivatives) is positive-definite at a point \(p,\) then the function is convex near p, and, conversely, if the function is convex near \(p,\) then the Hessian matrix is positive-semidefinite at \(p.\)

The set of positive definite matrices is an open convex cone, while the set of positive semi-definite matrices is a closed convex cone.

• The identity matrix \(I = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}\) is positive-definite (and as such also positive semi-definite). It is a real symmetric matrix, and, for any non-zero column vector z with real entries a and b, one has \[\mathbf{z}^\mathsf{T} I\mathbf{z} = \begin{bmatrix} a & b \end{bmatrix} \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} \begin{bmatrix} a \\ b \end{bmatrix} = a^2 + b^2.\] Seen as a complex matrix, for any non-zero column vector z with complex entries a and b one has \[\mathbf{z}^*I\mathbf{z} = \begin{bmatrix} \overline{a} & \overline{b} \end{bmatrix} \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} \begin{bmatrix} a \\ b\end{bmatrix} = \overline{a}a + \overline{b}b = |a|^2 + |b|^2.\] Either way, the result is positive since \(\mathbf z\) is not the zero vector (that is, at least one of \(a\) and \(b\) is not zero).
• The real symmetric matrix \[M = \begin{bmatrix} 2 & -1 & 0 \\ -1 & 2 & -1 \\ 0 & -1 & 2 \end{bmatrix}\] is positive-definite since for any non-zero column vector z with entries a, b and c, we have \[\begin{aligned} \mathbf{z}^\mathsf{T} M \mathbf{z} = \left( \mathbf{z}^\mathsf{T} M \right) \mathbf{z} &= \begin{bmatrix} (2a - b) & (-a + 2b - c) & (-b + 2c) \end{bmatrix} \begin{bmatrix} a \\ b \\ c \end{bmatrix} \\ &= (2a - b)a + (-a + 2b - c)b + (-b + 2c)c \\ &= 2a^2 - ba - ab + 2b^2 - cb - bc + 2c^2 \\ &= 2a^2 - 2ab + 2b^2 - 2bc + 2c^2 \\ &= a^2 + a^2 - 2ab + b^2 + b^2- 2bc + c^2 + c^2 \\ &= a^2 + (a - b)^2 + (b - c)^2 + c^2 \end{aligned}\] This result is a sum of squares, and therefore non-negative; and is zero only if \(a = b = c = 0,\) that is, when \(\mathbf{z}\) is the zero vector.
• For any real invertible matrix \(A,\) the product \(A^\mathsf{T} A\) is a positive definite matrix (if the means of the columns of A are 0, then this is also called the covariance matrix). A simple proof is that for any non-zero vector \(\mathbf{z},\) the condition \(\mathbf{z}^\mathsf{T} A^\mathsf{T} A\mathbf{z} = (A\mathbf{z})^\mathsf{T} (A\mathbf{z}) = \|A\mathbf{z}\|^2 > 0,\) since the invertibility of matrix \(A\) means that \(A\mathbf{z} \neq 0.\)
• The example \(M\) above shows that a matrix in which some elements are negative may still be positive definite. Conversely, a matrix whose entries are all positive is not necessarily positive definite, as for example \[N = \begin{bmatrix} 1 & 2 \\ 2 & 1 \end{bmatrix},\] for which \(\begin{bmatrix} -1 & 1 \end{bmatrix}N\begin{bmatrix} -1 & 1 \end{bmatrix}^\mathsf{T} = -2 < 0.\)

Let \(M\) be an \(n \times n\) Hermitian matrix (this includes real symmetric matrices). All eigenvalues of \(M\) are real, and their signs characterize its definiteness:

• \(M\) is positive definite if and only if all of its eigenvalues are positive.
• \(M\) is positive semi-definite if and only if all of its eigenvalues are non-negative.
• \(M\) is negative definite if and only if all of its eigenvalues are negative.
• \(M\) is negative semi-definite if and only if all of its eigenvalues are non-positive.
• \(M\) is indefinite if and only if it has both positive and negative eigenvalues.

Let \(P D P^{-1}\) be an eigendecomposition of \(M,\) where \(P\) is a unitary complex matrix whose columns comprise an orthonormal basis of eigenvectors of \(M,\) and \(D\) is a real diagonal matrix whose main diagonal contains the corresponding eigenvalues. The matrix \(M\) may be regarded as a diagonal matrix \(D\) that has been re-expressed in coordinates of the (eigenvectors) basis \(P.\) Put differently, applying \(M\) to some vector \(\mathbf{z},\) giving \(M \mathbf{z},\) is the same as changing the basis to the eigenvector coordinate system using \(P^{-1},\) giving \(P^{-1} \mathbf{z},\) applying the stretching transformation \(D\) to the result, giving \(D P^{-1} \mathbf{z},\) and then changing the basis back using \(P,\) giving \(P D P^{-1} \mathbf{z}.\)

With this in mind, the one-to-one change of variable \(\mathbf{y} = P\mathbf{z}\) shows that \(\mathbf{z}^* M\mathbf{z}\) is real and positive for any complex vector \(\mathbf{z}\) if and only if \(\mathbf{y}^* D \mathbf{y}\) is real and positive for any \(y;\) in other words, if \(D\) is positive definite. For a diagonal matrix, this is true only if each element of the main diagonal: that is, every eigenvalue of \(M\), is positive. Since the spectral theorem guarantees all eigenvalues of a Hermitian matrix to be real, the positivity of eigenvalues can be checked using Descartes' rule of alternating signs when the characteristic polynomial of a real, symmetric matrix \(M\) is available.

Let \(M\) be an \(n \times n\) Hermitian matrix. \(M\) is positive semidefinite if and only if it can be decomposed as a product \[M = B^* B\] of a matrix \(B\) with its conjugate transpose.

When \(M\) is real, \(B\) can be real as well and the decomposition can be written as \[M = B^\mathsf{T} B.\]

\(M\) is positive definite if and only if such a decomposition exists with \(B\) invertible. More generally, \(M\) is positive semidefinite with rank \(k\) if and only if a decomposition exists with a \(k \times n\) matrix \(B\) of full row rank (i.e. of rank \(k\)). Moreover, for any decomposition \(M = B^* B,\) \(\operatorname{rank}(M) = \operatorname{rank}(B).\)

If \(M = B^* B,\) then \(x^* M x = (x^* B^*) (B x) = \|B x \|^2 \geq 0,\) so \(M\) is positive semidefinite. If moreover \(B\) is invertible then the inequality is strict for \(x \neq 0,\) so \(M\) is positive definite. If \(B\) is \(k \times n\) of rank \(k,\) then \(\operatorname{rank}(M) = \operatorname{rank}(B^*) = k.\)

In the other direction, suppose \(M\) is positive semidefinite. Since \(M\) is Hermitian, it has an eigendecomposition \(M = Q^{-1} D Q\) where \(Q\) is unitary and \(D\) is a diagonal matrix whose entries are the eigenvalues of \(M\) Since \(M\) is positive semidefinite, the eigenvalues are non-negative real numbers, so one can define \(D^{\frac{1}{2}}\) as the diagonal matrix whose entries are non-negative square roots of eigenvalues. Then \(M = Q^{-1} D Q = Q^* D Q = Q^* D^{\frac{1}{2}} D^{\frac{1}{2}} Q = Q^* D^{\frac{1}{2}*} D^{\frac{1}{2}} Q = B^* B\) for \(B = D^{\frac{1}{2}} Q.\) If moreover \(M\) is positive definite, then the eigenvalues are (strictly) positive, so \(D^{\frac{1}{2}}\) is invertible, and hence \(B = D^{\frac{1}{2}} Q\) is invertible as well. If \(M\) has rank \(k,\) then it has exactly \(k\) positive eigenvalues and the others are zero, hence in \(B = D^{\frac{1}{2}} Q\) all but \(k\) rows are all zeroed. Cutting the zero rows gives a \(k \times n\) matrix \(B'\) such that \(B'^* B' = B^* B = M.\)

The columns \(b_1, \dots, b_n\) of \(B\) can be seen as vectors in the complex or real vector space \(\mathbb{R}^k,\) respectively. Then the entries of \(M\) are inner products (that is dot products, in the real case) of these vectors \[M_{ij} = \langle b_i, b_j\rangle.\] In other words, a Hermitian matrix \(M\) is positive semidefinite if and only if it is the Gram matrix of some vectors \(b_1, \dots, b_n.\) It is positive definite if and only if it is the Gram matrix of some linearly independent vectors. In general, the rank of the Gram matrix of vectors \(b_1, \dots, b_n\) equals the dimension of the space spanned by these vectors.

The decomposition is not unique: if \(M = B^* B\) for some \(k \times n\) matrix \(B\) and if \(Q\) is any unitary \(k \times k\) matrix (meaning \(Q^* Q = Q Q^* = I\)), then \(M = B^* B = B^* Q^* Q B = A^* A\) for \(A = Q B.\)

However, this is the only way in which two decompositions can differ: The decomposition is unique up to unitary transformations. More formally, if \(A\) is a \(k \times n\) matrix and \(B\) is a \(\ell \times n\) matrix such that \(A^* A = B^* B,\) then there is a \(\ell \times k\) matrix \(Q\) with orthonormal columns (meaning \(Q^* Q = I_{k \times k}\)) such that \(B = Q A.\) When \(\ell = k\) this means \(Q\) is unitary.

This statement has an intuitive geometric interpretation in the real case: let the columns of \(A\) and \(B\) be the vectors \(a_1,\dots,a_n\) and \(b_1, \dots, b_n\) in \(\mathbb{R}^k.\) A real unitary matrix is an orthogonal matrix, which describes a rigid transformation (an isometry of Euclidean space \(\mathbb{R}^k\)) preserving the 0 point (i.e. rotations and reflections, without translations). Therefore, the dot products \(a_i \cdot a_j\) and \(b_i \cdot b_j\) are equal if and only if some rigid transformation of \(\mathbb{R}^k\) transforms the vectors \(a_1,\dots,a_n\) to \(b_1,\dots,b_n\) (and 0 to 0).

A Hermitian matrix \(M\) is positive semidefinite if and only if there is a positive semidefinite matrix \(B\) (in particular \(B\) is Hermitian, so \(B^* = B\)) satisfying \(M = B B.\) This matrix \(B\) is unique, is called the non-negative square root of \(M,\) and is denoted with \(B = M^\frac{1}{2}.\) When \(M\) is positive definite, so is \(M^\frac{1}{2},\) hence it is also called the positive square root of \(M .\)

The non-negative square root should not be confused with other decompositions \(M = B^* B.\) Some authors use the name square root and \(M^\frac{1}{2}\) for any such decomposition, or specifically for the Cholesky decomposition, or any decomposition of the form \(M = B B;\) others only use it for the non-negative square root.

If \(M \succ N \succ 0\) then \(M^\frac{1}{2} \succ N^\frac{1}{2} \succ 0.\)

A Hermitian positive semidefinite matrix \(M\) can be written as \(M = L L^*,\) where \(L\) is lower triangular with non-negative diagonal (equivalently \(M = B^*B\) where \(B = L^*\) is upper triangular); this is the Cholesky decomposition. If \(M\) is positive definite, then the diagonal of \(L\) is positive and the Cholesky decomposition is unique. Conversely if \(L\) is lower triangular with nonnegative diagonal then \(L L^*\) is positive semidefinite. The Cholesky decomposition is especially useful for efficient numerical calculations. A closely related decomposition is the LDL decomposition, \(M = L D L^*,\) where \(D\) is diagonal and \(L\) is lower unitriangular.

Any \(2n\times 2n\) positive definite Hermitian real matrix \(M\) can be diagonalized via symplectic (real) matrices. More precisely, Williamson's theorem ensures the existence of symplectic \(S\in\mathbf{Sp}(2n,\mathbb{R})\) and diagonal real positive \(D\in\mathbb{R}^{n\times n}\) such that \(SMS^T=D\oplus D\).