maths.free › Linear Algebra › Geometry and spectra › Singular value decomposition
Singular value decomposition
In linear algebra, the singular value decomposition (SVD) is a factorization of a real or complex matrix into a rotation, followed by a scaling, followed by another rotation.
Singular value decomposition
In linear algebra, the singular value decomposition (SVD) is a factorization of a real or complex matrix into a rotation, followed by a scaling, followed by another rotation. It generalizes the eigendecomposition of a square normal matrix with an orthonormal eigenbasis to any \(m \times n\) matrix. It is related to the polar decomposition, and is a common means for implementing low-rank approximation for matrices.
Specifically, the singular value decomposition of an \(m \times n\) complex matrix \(\mathbf M\) is a factorization of the form \(\mathbf{M} = \mathbf{U} \mathbf{\Sigma} \mathbf{V}^*\), where \(\mathbf U\) is an \(m \times m\) complex unitary matrix, \(\mathbf \Sigma\) is an \(m \times n\) rectangular diagonal matrix with non-negative real numbers on the diagonal, \(\mathbf V\) is an \(n \times n\) complex unitary matrix, and \(\mathbf{V}^*\) is the conjugate transpose of \(\mathbf V\). Such decompositions always exist for any complex matrix. If \(\mathbf M\) is real, then some \(\mathbf U\) and \(\mathbf V\) can be found which are real (orthogonal) matrices; a real-valued SVD is often denoted \(\mathbf{U} \mathbf{\Sigma} \mathbf{V}^\mathsf{T}\), where \(\mathbf{V}^\mathsf{T}\) is the transpose of \(\mathbf V\).
The diagonal entries \(\sigma_i = \mathbf{\Sigma}_{i,i}\) of \(\mathbf \Sigma\) are uniquely determined by \(\mathbf M\), up to reordering, and are known as the singular values of \(\mathbf M\). Conventionally they are arranged in descending order (from largest to smallest), which uniquely determines \(\mathbf \Sigma\). The number of non-zero singular values, allowing repetitions, is equal to \(r\), the rank of \(\mathbf M\).
The columns of \(\mathbf U\) and the columns of \(\mathbf V\) are called left-singular vectors and right-singular vectors of \(\mathbf M\), respectively. They form two orthonormal bases, \(\{ \mathbf{u}_1, \ldots, \mathbf{u}_m \}\) and \(\{ \mathbf{v}_1, \ldots, \mathbf{v}_n \}\). In general the SVD is not unique, with certain unitary transformations of \(\mathbf U\) and \(\mathbf V\) producing valid alternative decompositions.
The term SVD sometimes refers to the compact SVD, a similar decomposition \(\mathbf{M} = \mathbf{U}_r \mathbf{\Sigma}_r \mathbf{V}_r^*\), in which \(\mathbf{\Sigma}_r\) is an \(r \times r\) matrix with only the non-zero singular values (allowing repetitions) on its main diagonal. In this variant, \(\mathbf{U}_r\) is an \(m \times r\) semi-unitary matrix whose columns \(\{ \mathbf{u}_1, \ldots, \mathbf{u}_r \}\) span the columns of \(\mathbf M\), and \(\mathbf{V}_r\) is an \(n \times r\) semi-unitary matrix whose columns \(\{ \mathbf{v}_1, \ldots, \mathbf{v}_r \}\) span the columns of \(\mathbf{M}^*\!\).
The SVD (with sorted singular values) splits \(\mathbf M\) into a sum of \(r\) rank-\(1\) matrices, \(\textstyle \mathbf{M} = \sigma_1 \mathbf{u}_1 \mathbf{v}_1^* + \sigma_2 \mathbf{u}_2 \mathbf{v}_2^* + \cdots + \sigma_r \mathbf{u}_r \mathbf{v}_r^*\!\).
Condensed: the full section is in Wikipedia.
Rotation/reflection, coordinate scaling, rotation/reflection
In the special case when \(\mathbf M\) is an \(m \times m\) real square matrix, the matrices \(\mathbf U\) and \(\mathbf{V}^*\) can be chosen to be real orthogonal \(m \times m\) matrices. \(\mathbf M\) can be interpreted to represent a linear transformation \(\mathbf{x} \mapsto \mathbf{M} \mathbf{x}\) of the Euclidean space \(\R^m\); then the matrices \(\mathbf U\) and \(\mathbf{V}^*\) represent rotations or reflections of the space, while \(\mathbf \Sigma\) represents the scaling of each coordinate \(\mathbf{x}_i\) by the factor \(\sigma_i\). Thus the SVD decomposition breaks down any linear transformation of \(\R^m\) into a composition of three geometric transformations: a rotation or reflection \(\mathbf{V}^*\) followed by a coordinate-by-coordinate scaling \(\mathbf \Sigma\) followed by another rotation or reflection \(\mathbf U\).
In particular, if \(\mathbf M\) has a positive determinant, then \(\mathbf U\) and \(\mathbf{V}^*\) can be chosen to be both rotations with reflections, or both rotations without reflections. If the determinant is negative, exactly one of them will have a reflection. If the determinant is zero, each can be independently chosen to be of either type.
In the more general case when the matrix \(\mathbf M\) is real but not square, namely \(m \times n\) with \(m \neq n\), it can be interpreted as a linear transformation from \(\R^n\) to \(\R^m\). Then \(\mathbf U\) and \(\mathbf{V}^*\) can be chosen to be rotations/reflections of \(\R^m\) and \(\R^n\), respectively; and \(\mathbf \Sigma\), besides scaling the first \(\min \{m,n\}\) coordinates, also extends the vector with zeros and Σ, i.e. removes trailing coordinates, so as to turn \(\R^n\) into \(\R^m\).
Singular values as semiaxes of an ellipse or ellipsoid
As shown in the figure, the singular values can be interpreted as the magnitude of the semiaxes of an ellipse in 2D. This concept can be generalized to \(n\)-dimensional Euclidean space, with the singular values of any \(n \times n\) square matrix being viewed as the magnitude of the semiaxis of an \(n\)-dimensional ellipsoid. Similarly, the singular values of any \(m \times n\) matrix can be viewed as the magnitude of the semiaxis of an \(n\)-dimensional ellipsoid in \(m\)-dimensional space, for example as an ellipse in a (tilted) 2D plane in a 3D space. Singular values encode magnitude of the semiaxis, while singular vectors encode direction. See below for further details.
The columns of U and V are orthonormal bases
Since \(\mathbf U\) and \(\mathbf V^*\) are unitary, the columns of each of them form a set of orthonormal vectors, which can be regarded as basis vectors. The matrix \(\mathbf M\) maps the basis vector \(\mathbf v_i\) to the stretched unit vector \(\sigma_i \mathbf u_i\). By the definition of a unitary matrix, the same is true for their conjugate transposes \(\mathbf U^*\) and \(\mathbf V\), except the geometric interpretation of the singular values as stretches is lost. In short, the columns of \(\mathbf U\), \(\mathbf U^*\), \(\mathbf V\), and \(\mathbf V^*\) are orthonormal bases. When \(\mathbf M\) is a positive-semidefinite Hermitian matrix, \(\mathbf U\) and \(\mathbf V\) are both equal to the unitary matrix used to diagonalize \(\mathbf M\). However, when \(\mathbf M\) is not positive-semidefinite and Hermitian but still diagonalizable, its eigendecomposition and singular value decomposition are distinct.
Relation to the four fundamental subspaces
- The first \(r\) columns of \(\mathbf U\) are a basis of the column space of \(\mathbf M\).
- The last \(m-r\) columns of \(\mathbf U\) are a basis of the null space of \(\mathbf M^*\).
- The first \(r\) columns of \(\mathbf V\) are a basis of the column space of \(\mathbf M^*\) (the row space of \(\mathbf M\) in the real case).
- The last \(n-r\) columns of \(\mathbf V\) are a basis of the null space of \(\mathbf M\).
Geometric meaning
Because \(\mathbf U\) and \(\mathbf V\) are unitary, the columns \(\mathbf{u}_1, \ldots, \mathbf{u}_m\) of \(\mathbf U\) are an orthonormal basis of \(\C^m\) and the columns \(\mathbf{v}_1, \ldots, \mathbf{v}_n\) of \(\mathbf V\) are an orthonormal basis of \(\C^n\) (with respect to the standard scalar products on these spaces).
The linear transformation \[T \colon \left \{\begin{align} \C^n & \to \C^m \\ \mathbf{x} & \mapsto \mathbf{Mx} \end{align} \right.\] has a particularly simple description with respect to these orthonormal bases: we have \[T(\mathbf{v}_i) = \sigma_i \mathbf{u}_i, \qquad i = 1, \ldots, \min \{m, n\},\] where \(\sigma_i\) is the \(i\)-th diagonal entry of \(\mathbf \Sigma\), and \(T(\mathbf{v}_i) = \mathbf 0\) for \(i > \min \{m,n\}\).
The geometric content of the SVD theorem can thus be summarized as follows: for every linear map \(T \colon \C^n \to \C^m\) one can find orthonormal bases of \(\C^n\) and \(\C^m\) such that \(T\) maps the \(i\)-th basis vector of \(\C^n\) to a non-negative multiple of the \(i\)-th basis vector of \(\C^m\), and sends the leftover basis vectors to zero. With respect to these bases, the map \(T\) is therefore represented by a diagonal matrix with non-negative real diagonal entries.
To get a more visual flavor of singular values and SVD factorization, at least when working on real vector spaces, consider the unit sphere \(S\) in \(\R^n\). The linear map \(T\) maps this sphere onto an ellipsoid in \(\R^m\). Non-zero singular values are simply the lengths of the semi-axes of this ellipsoid. Especially when \(n = m\), and all the singular values are distinct and non-zero, the SVD of the linear map \(T\) can be easily analyzed as a succession of three consecutive moves: consider the ellipsoid \(T(S)\) and specifically its axes; then consider the directions in \(\R^n\) sent by \(T\) onto these axes. These directions happen to be mutually orthogonal. Apply first an isometry \(\mathbf{V}^*\) sending these directions to the coordinate axes of \(\R^n\). On a second move, apply an endomorphism \(\mathbf D\) diagonalized along the coordinate axes and stretching or shrinking in each direction, using the semi-axis lengths of \(T(S)\) as scaling coefficients. The composition \(\mathbf{D} \circ \mathbf{V}^*\) then sends the unit-sphere onto an ellipsoid isometric to \(T(S)\). To define the third and last move, apply an isometry \(\mathbf U\) to this ellipsoid to obtain \(T(S)\). As can be easily checked, the composition \(\mathbf{U} \circ \mathbf{D} \circ \mathbf{V}^*\) coincides with \(T\).
Example
For example, the following \(4 \times 5\) matrix \(\mathbf M\) can be decomposed as \(\mathbf{U \Sigma V}^*\): \[\begin{align} \mathbf{M} &= \begin{bmatrix} ~4 & ~8 & ~8 & ~0 & ~0 \\ -3 & -6 & -6 & ~0 & ~0 \\ -4 & ~4 & -2 & ~0 & ~0 \\ ~0 & ~0 & ~0 & -4 & ~3 \end{bmatrix} \\ &= \,\underbrace{\!\begin{bmatrix} \color{PineGreen}~\frac45 & \color{BrickRed}~0 & \color{BlueViolet}~0 & \color{CadetBlue}~\frac35 \\ \color{PineGreen}-\frac35 & \color{BrickRed}~0 & \color{BlueViolet}~0 & \color{CadetBlue}~\frac45 \\ \color{PineGreen}~0\vphantom{\frac11} & \color{BrickRed}~1 & \color{BlueViolet}~0 & \color{CadetBlue}~0 \\ \color{PineGreen}~0\vphantom{\frac11} & \color{BrickRed}~0 & \color{BlueViolet}~1 & \color{CadetBlue}~0 \end{bmatrix}\!}_{\textstyle \mathbf{U}}\, \,\underbrace{\!\begin{bmatrix} \color{PineGreen}15\vphantom{\frac11} & \color{Gray}0 & \color{Gray}0 & \color{Gray}0 & \color{Gray}0 \\ \color{Gray}0\vphantom{\frac11} & \color{BrickRed}6 & \color{Gray}0 & \color{Gray}0 & \color{Gray}0 \\ \color{Gray}0\vphantom{\frac11} & \color{Gray}0 & \color{BlueViolet}5 & \color{Gray}0 & \color{Gray}0 \\ \color{Gray}0\vphantom{\frac11} & \color{Gray}0 & \color{Gray}0 & \color{CadetBlue}0 & \color{Gray}0 \end{bmatrix}\!}_{\textstyle \mathbf{\Sigma}}\, \,\underbrace{\!\begin{bmatrix} \color{PineGreen}~\frac13 & \color{PineGreen}~\frac23 & \color{PineGreen}~\frac23 & \color{PineGreen}~0 & \color{PineGreen}~0 \\ \color{BrickRed}-\frac23 & \color{BrickRed}~\frac23 & \color{BrickRed}-\frac13 & \color{BrickRed}~0 & \color{BrickRed}~0 \\ \color{BlueViolet}~0 & \color{BlueViolet}~0 & \color{BlueViolet}~0 & \color{BlueViolet}-\frac45 & \color{BlueViolet}~\frac35 \\ \color{CadetBlue}-\frac23 & \color{CadetBlue}-\frac13 & \color{CadetBlue}~\frac23 & \color{CadetBlue}~0 & \color{CadetBlue}~0 \\ \color{CadetBlue}~0 & \color{CadetBlue}~0 & \color{CadetBlue}~0 & \color{CadetBlue}~\frac35 & \color{CadetBlue}~\frac45 \end{bmatrix}\!}_{\textstyle \mathbf{V}^*}\,. \end{align}\]
The singular values of \(\mathbf M\) are the diagonal entries of \(\mathbf \Sigma\): \(\color{PineGreen}15\), \(\color{BrickRed}6\), \(\color{BlueViolet}5\), \(\color{CadetBlue}0\). The corresponding left- and right-singular vectors are the columns \(\mathbf u_j\) of \(\mathbf U\) and rows \(\mathbf v_j^*\) of \(\mathbf V^*\), respectively. That is, \[\begin{align} \mathbf{M} {\color{PineGreen}\mathbf{v}_1} &= {\color{PineGreen}15 \mathbf{u}_1}, & {\color{PineGreen}\mathbf{u}_1^*} \mathbf{M} &= {\color{PineGreen}15 \mathbf{v}_1^*}, \\ \mathbf{M} {\color{BrickRed}\mathbf{v}_2} &= {\color{BrickRed}6 \mathbf{u}_2}, & {\color{BrickRed}\mathbf{u}_2^*} \mathbf{M} &= {\color{BrickRed}6 \mathbf{v}_2^*}, \\ \mathbf{M} {\color{BlueViolet}\mathbf{v}_3} &= {\color{BlueViolet}5 \mathbf{u}_3}, & {\color{BlueViolet}\mathbf{u}_3^*} \mathbf{M} &= {\color{BlueViolet}5 \mathbf{v}_3^*}, \\ \mathbf{M} {\color{CadetBlue}\mathbf{v}_4} &= {\color{CadetBlue}0 \mathbf{u}_4} = {\color{CadetBlue}\mathbf{0}}, & {\color{CadetBlue}\mathbf{u}_4^*} \mathbf{M} &= {\color{CadetBlue}0 \mathbf{v}_4^*} = {\color{CadetBlue}\mathbf{0}}, \\ \mathbf{M} {\color{CadetBlue}\mathbf{v}_5} &= {\color{CadetBlue}\mathbf{0}}. \end{align}\]
Condensed: the full section is in Wikipedia.
Singular values, singular vectors, and their relation to the SVD
A non-negative real number \(\sigma\) is a singular value for \(\mathbf M\) if and only if there exist unit vectors \(\mathbf u\) in \(\C^m\) and \(\mathbf v\) in \(\C^n\) such that \[\begin{align} \mathbf{M v} &= \sigma \mathbf{u}, \\[3mu] \mathbf{M}^* \mathbf{u} &= \sigma \mathbf{v}. \end{align}\]
The vectors \(\mathbf u\) and \(\mathbf v\) are called left-singular and right-singular vectors for \(\sigma\), respectively.
In any singular value decomposition \(\mathbf{M} = \mathbf{U \Sigma V}^*\), the diagonal entries of \(\mathbf \Sigma\) comprise the singular values of \(\mathbf M\). The first \(p = \min \{ m,n \}\) columns of \(\mathbf U\) and \(\mathbf V\) are, respectively, left- and right-singular vectors for the corresponding singular values. Consequently,
- An \(m \times n\) matrix \(\mathbf M\) has at most \(p\) distinct singular values.
- It is always possible to find a unitary basis \(\mathbf U\) for \(\C^m\) with a subset of basis vectors spanning the left-singular vectors of each singular value of \(\mathbf M\).
- It is always possible to find a unitary basis \(\mathbf V\) for \(\C^n\) with a subset of basis vectors spanning the right-singular vectors of each singular value of \(\mathbf M\).
The diagonal entries of \(\Sigma\) need not be distinct. A singular value \(\sigma\) which appears among them \(k\) times has a \(k\)-dimensional subspace of corresponding left-singular vectors. Any orthonormal basis for this subspace can be taken as the corresponding columns of some matrix \(\mathbf U\), and uniquely determines an orthonormal basis for the \(k\)-dimensional subspace of right-singular vectors which form the corresponding columns of a matrix \(\mathbf V\). When \(\sigma \neq 0\) and \(k = 1\), the singular value is called non-degenerate, and the corresponding left-singular vector and right-singular vector are unique up to multiplication by a phase factor (a unit complex number), or, in the real case, unique up to sign. When \(k > 1\), the singular value \(\sigma\) is called degenerate.
The left- and right-singular vectors of singular value \(0\) comprise all unit vectors in the cokernel and kernel, respectively, of \(\mathbf M\). By the rank, nullity theorem, these subspaces cannot have the same dimension if \(m \neq n\). Even when all singular values are non-zero, if \(m > n\), then the cokernel is non-trivial, in which case \(\mathbf U\) is padded with \(m - n\) orthogonal unit vectors from the cokernel. Conversely, if \(m < n\), then \(\mathbf V\) is padded by \(n - m\) orthogonal unit vectors from the kernel. However, if the singular value \(0\) exists, the extra columns of \(\mathbf U\) or \(\mathbf V\) already appear as left- or right-singular vectors.[…, if the singular value 0 exists, the extra columns of U or V must not repeat the corresponding left- or right-singular vectors.] [Otherwise, U or V would not have column full rank.]
If all singular values of a square matrix \(\mathbf M\) are non-zero and non-degenerate, then its singular value decomposition is unique up to multiplication of any column of \(\mathbf U\) and the corresponding column of \(\mathbf V\) by an arbitrary phase factor. More generally, the SVD of any matrix \(\mathbf M\) is unique up to unitary transformations applied uniformly to the column vectors of both \(\mathbf U\) and \(\mathbf V\) (and which do not mix columns corresponding to distinct singular values).
Relation to eigenvalue decomposition
The singular value decomposition is very general in the sense that it can be applied to any \(m \times n\) matrix, whereas eigenvalue decomposition can only be applied to square diagonalizable matrices. Nevertheless, the two decompositions are related.
If \(\mathbf M\) has SVD \(\mathbf{M} = \mathbf{U} \mathbf{\Sigma} \mathbf{V}^*\), the following two relations hold: \[\begin{align} \mathbf{M}^* \mathbf{M} &= \mathbf{V} \mathbf{\Sigma}^* \mathbf{U}^*\, \mathbf{U} \mathbf{\Sigma} \mathbf{V}^* = \mathbf{V} (\mathbf{\Sigma}^* \mathbf{\Sigma}) \mathbf{V}^*, \\[3mu] \mathbf{M} \mathbf{M}^* &= \mathbf{U} \mathbf{\Sigma} \mathbf{V}^*\, \mathbf{V} \mathbf{\Sigma}^* \mathbf{U}^* = \mathbf{U} (\mathbf{\Sigma} \mathbf{\Sigma}^*) \mathbf{U}^*. \end{align}\]
The right-hand sides of these relations describe the eigenvalue decompositions of the left-hand sides. Consequently:
- The columns of \(\mathbf V\) (referred to as right-singular vectors) are eigenvectors of \(\mathbf{M}^* \mathbf{M}\).
- The columns of \(\mathbf U\) (referred to as left-singular vectors) are eigenvectors of \(\mathbf{M} \mathbf{M}^*\).
- The non-zero elements of \(\mathbf \Sigma\) (non-zero singular values) are the square roots of the non-zero eigenvalues of \(\mathbf{M}^* \mathbf{M}\) or \(\mathbf{M} \mathbf{M}^*\).
In the special case of \(\mathbf M\) being a normal matrix, and thus also square, the spectral theorem ensures that it can be unitarily diagonalized using a basis of eigenvectors, and thus decomposed as \(\mathbf{M} = \mathbf{U} \mathbf{D} \mathbf{U}^*\) for some unitary matrix \(\mathbf U\) and diagonal matrix \(\mathbf D\) with complex elements \(\sigma_i\) along the diagonal. When \(\mathbf M\) is positive semi-definite, the \(\sigma_i\) will be non-negative real numbers, so the decomposition \(\mathbf{M} = \mathbf{U} \mathbf{D} \mathbf{U}^*\) is also a singular value decomposition. Otherwise, it can be recast as an SVD by moving the phase \(e^{i \varphi}\) of each \(\sigma_i\) to either its corresponding \(\mathbf{v}_i\) or \(\mathbf{u}_i\). The natural connection of the SVD to non-normal matrices is through the polar decomposition theorem: \(\mathbf{M} = \mathbf{S} \mathbf{R}\), where \(\mathbf{S} = \mathbf{U} \mathbf{\Sigma} \mathbf{U}^*\) is positive semidefinite and normal, and \(\mathbf{R} = \mathbf{U} \mathbf{V}^*\) is unitary.
Thus, except for positive semi-definite matrices, the eigenvalue decomposition and SVD of \(\mathbf M\), while related, differ: the eigenvalue decomposition is \(\mathbf{M} = \mathbf{U} \mathbf{D} \mathbf{U}^{-1}\), where \(\mathbf U\) is not necessarily unitary and \(\mathbf D\) is not necessarily positive semi-definite, while the SVD is \(\mathbf{M} = \mathbf{U} \mathbf{\Sigma} \mathbf{V}^*\), where \(\mathbf \Sigma\) is diagonal and positive semi-definite, and \(\mathbf U\) and \(\mathbf V\) are unitary matrices that are not necessarily related except through the matrix \(\mathbf M\). While only non-defective square matrices have an eigenvalue decomposition, any \(m \times n\) matrix has an SVD.
Pseudoinverse
The singular value decomposition can be used for computing the pseudoinverse of a matrix. The pseudoinverse of the matrix \(\mathbf M\) with singular value decomposition \(\mathbf{M} = \mathbf{U \Sigma V}^*\) is \[\mathbf{M}^+ = \mathbf{V} \mathbf{\Sigma}^+ \mathbf{U}^*,\] where \(\mathbf{\Sigma}^+\) is the pseudoinverse of \(\mathbf \Sigma\), which is formed by replacing every non-zero diagonal entry of \(\mathbf \Sigma\) by its reciprocal and transposing the resulting matrix. The pseudoinverse is one way to solve linear least squares problems.
Solving homogeneous linear equations
A set of homogeneous linear equations can be written as \(\mathbf A \mathbf x = \mathbf 0\) for a matrix \(\mathbf A\), a vector \(\mathbf x\), and the zero vector \(\mathbf 0\). A typical situation is that \(\mathbf A\) is known and a non-zero \(\mathbf x\) is to be determined which satisfies the equation. Such an \(\mathbf x\) belongs to \(\mathbf A\)'s null space and is sometimes called a (right) null vector of \(\mathbf A\). The vector \(\mathbf x\) can be characterized as a right-singular vector corresponding to a singular value of \(\mathbf A\) that is zero. This observation means that if \(\mathbf A\) is a square matrix and has no vanishing singular value, the equation has no non-zero \(\mathbf x\) as a solution. It also means that if there are several vanishing singular values, any linear combination of the corresponding right-singular vectors is a valid solution. Analogously to the definition of a (right) null vector, a non-zero \(\mathbf x\) satisfying \(\mathbf{x}^* \mathbf{A} = \mathbf 0\), where \(\mathbf{x}^*\) denotes the conjugate transpose of \(\mathbf x\), is called a left null vector of \(\mathbf A\).
Total least squares minimization
A total least squares problem seeks the vector \(\mathbf x\) that minimizes the 2-norm of a vector \(\mathbf A \mathbf x\) under the constraint \(\| \mathbf{x} \| = 1\). The solution turns out to be the right-singular vector of \(\mathbf A\) corresponding to the smallest singular value.
Range, null space, and rank
Another application of the SVD is that it provides an explicit representation of the range and null space of a matrix \(\mathbf M\). The right-singular vectors corresponding to vanishing singular values of \(\mathbf M\) span the null space of \(\mathbf M\) and the left-singular vectors corresponding to the non-zero singular values of \(\mathbf M\) span the range of \(\mathbf M\).
As a consequence, the rank of \(\mathbf M\) equals the number of non-zero singular values, which is the same as the number of non-zero diagonal elements in \(\mathbf \Sigma\). In numerical linear algebra, the singular values can be used to determine the effective rank of a matrix, as rounding error may lead to small but non-zero singular values in a rank-deficient matrix. Singular values beyond a significant gap are assumed to be numerically equivalent to zero.
Low-rank matrix approximation
Some practical applications need to solve the problem of approximating a matrix \(\mathbf M\) with another matrix \(\tilde{\mathbf M}\), said to be truncated, which has a specific rank \(t\). In the case that the approximation is based on minimizing the Frobenius norm of the difference between \(\mathbf M\) and \(\tilde{\mathbf M}\) under the constraint that \(\operatorname{rank} \! \bigl( \tilde{\mathbf M} \bigr) = t\), it turns out that the solution is given by the SVD of \(\mathbf M\), namely \[\tilde{\mathbf{M}} = \mathbf{U} \tilde{\mathbf \Sigma} \mathbf{V}^*,\] where \(\tilde{\mathbf \Sigma}\) is the same matrix as \(\mathbf \Sigma\) except that it contains only the \(t\) largest singular values (the other singular values are replaced by zero). Or, equivalently, by the truncated SVD. This is known as the Eckart-Young theorem, as it was proved by those two authors in 1936.
Tani ti. Asnjë kalkulator nuk e zgjidh këtë, por pjesët e saj janë të llogaritura. Provo një më poshtë, ose shkruaj tënde.
Një llogari e lirë shtohet shënime në çdo mësim, një regjistrim të asaj që ju keni përfunduar, problemet tuaja të zgjidhura në një vend, dhe një mësues që ju mund të pyesni rreth kësaj faqeje. Matematika vetë është e hapur për të gjithë, të regjistruar apo jo.
Regjistrohu HyrSimbolet e përdorura këtu
Prek çdo simbol për përkufizimin e plotë, një fotografi dhe se çfarë do të thotë çdo shkronjë në të.
Pyetja që bëjnë njerëzit
What does a determinant mean geometrically?
It is the factor by which the matrix scales area (2×2) or volume (3×3), with a negative sign if orientation flips. Zero means the matrix flattens space and cannot be undone.
What is an eigenvector?
A direction the matrix does not turn: it only stretches it by the eigenvalue. Along eigenvectors a complicated matrix acts like multiplication by a number.
Why is matrix multiplication not commutative?
Because a matrix is a transformation and AB means "do B, then A". Rotating then reflecting is not the same as reflecting then rotating.
Pjesa e kësaj faqeje është adaptuar nga Wikipedia (CC BY-SA 4.0). E përmbledhur dhe ri-shkruar këtu; gabimet janë tona.
Më shumë në Linear Algebra
DeterminantsMatrix inverseEigenvalues and eigenvectorsMatrix multiplicationRow reductionVector spaces, span and linear independenceOrthogonality, projections and least squaresDiagonalisation and matrix powersLinear transformations and change of basis