Hilbert space

The mathematical concept of a Hilbert space generalizes the notion of Euclidean space. It extends the methods of Euclidean geometry and calculus from the two-dimensional Euclidean plane and three-dimensional space to spaces of any finite or infinite dimension. A Hilbert space is an abstract vector space, and it has the additional structure of an inner product that allows length and angle to be measured. Finally, Hilbert spaces are required to be complete, a property that stipulates the existence of enough limits in the space to allow the techniques of calculus to be used.

Hilbert spaces were studied beginning in the first decade of the 20th century by David Hilbert (after whom they are named), Erhard Schmidt, and Frigyes Riesz. They are indispensable tools in the theories of partial differential equations, quantum mechanics, Fourier analysis (which includes applications to signal processing and heat transfer), and ergodic theory (which forms the mathematical underpinning of thermodynamics). John von Neumann coined the term Hilbert space for the abstract concept that underlies many of these diverse applications. The success of Hilbert space methods ushered in a very fruitful era for functional analysis. Apart from the classical Euclidean vector spaces, examples of Hilbert spaces include spaces of square-integrable functions, spaces of sequences, Sobolev spaces consisting of generalized functions, and Hardy spaces of holomorphic functions.

Geometric intuition plays an important role in many aspects of Hilbert space theory. Exact analogs of the Pythagorean theorem and parallelogram law hold in a Hilbert space. At a deeper level, perpendicular projection onto a linear subspace plays a significant role in optimization problems and other aspects of the theory. An element of a Hilbert space can be uniquely specified by its coordinates with respect to an orthonormal basis, in analogy with Cartesian coordinates in classical geometry. When this basis is countably infinite, it allows identifying the Hilbert space with the space of the infinite sequences that are square-summable. The latter space is often in the older literature referred to as the Hilbert space.

One of the most familiar examples of a Hilbert space is the Euclidean vector space consisting of three-dimensional vectors, denoted by \(\mathbf{R}^3\), and equipped with the dot product. The dot product takes two vectors x and y, and produces a real number x ⋅ y. If x and y are represented in Cartesian coordinates, then the dot product is defined by: \[\begin{pmatrix} x_1 \\ x_2 \\ x_3 \end{pmatrix} \cdot \begin{pmatrix} y_1 \\ y_2 \\ y_3 \end{pmatrix} = x_1 y_1 + x_2 y_2 + x_3 y_3 \,.\]

The dot product satisfies the properties:

1. It is symmetric in x and y: x ⋅ y = y ⋅ x.
2. It is linear in its first argument: (ax₁ + bx₂) ⋅ y = a(x₁ ⋅ y) + b(x₂ ⋅ y) for any scalars a, b, and vectors x₁, x₂, and y.
3. It is positive definite: for all vectors x, x ⋅ x ≥ 0 , with equality if and only if x = 0.

An operation on pairs of vectors that, like the dot product, satisfies these three properties is known as a (real) inner product. A vector space equipped with such an inner product is known as a (real) inner product space. Every finite-dimensional inner product space is also a Hilbert space. The basic feature of the dot product that connects it with Euclidean geometry is that it is related to both the length (or norm) of a vector, denoted ‖x‖, and to the angle θ between two vectors x and y by means of the formula \[\mathbf{x}\cdot\mathbf{y} = \left\|\mathbf{x}\right\| \left\|\mathbf{y}\right\| \, \cos\theta \,.\]

Multivariable calculus in Euclidean space relies on the ability to compute limits, and to have useful criteria for concluding that limits exist. A mathematical series \[\sum_{n=0}^\infty \mathbf{x}_n\] consisting of vectors in R is absolutely convergent provided that the sum of the lengths converges as an ordinary series of real numbers: \[\sum_{k=0}^\infty \|\mathbf{x}_k\| < \infty \,.\] Just as with a series of scalars, a series of vectors that converges absolutely also converges to some limit vector L in the Euclidean space, in the sense that \[\lim_{N\to\infty}\Biggl\| \mathbf{L} - \sum_{k=0}^N \mathbf{x}_k \Biggr\|=0.\] This property expresses the completeness of Euclidean space: that a series that converges absolutely also converges in the ordinary sense.

Hilbert spaces are often taken over the complex numbers. The complex plane denoted by C is equipped with a notion of magnitude, the complex modulus |z|, which is defined as the square root of the product of z with its complex conjugate: \[|z|^2 = z\overline{z} \,.\]

If z = x + iy is a decomposition of z into its real and imaginary parts, then the modulus is the usual Euclidean two-dimensional length: \[|z| = \sqrt{x^2 + y^2} \,.\]

The inner product of a pair of complex numbers z and w is the product of z with the complex conjugate of w: \[\langle z, w\rangle = z\overline{w}\,.\]

Condensed: the full section is in Wikipedia.

A Hilbert space is a real or complex inner product space that is also a complete metric space with respect to the distance function induced by the inner product.

To say that a complex vector space H is a complex inner product space means that there is an inner product \(\langle x, y \rangle\) associating a complex number to each pair of elements \(x, y\) of H that satisfies the following properties:

1. The inner product is conjugate symmetric; that is, the inner product of a pair of elements is equal to the complex conjugate of the inner product of the swapped elements: \[\langle y, x\rangle = \overline{\langle x, y\rangle}\,.\] Importantly, this implies that \(\langle x, x\rangle\) is a real number.
2. The inner product is linear in its first argument. For all complex numbers \(a\) and \(b,\) \[\langle ax_1 + bx_2, y\rangle = a\langle x_1, y\rangle + b\langle x_2, y\rangle\,.\]
3. The inner product of an element with itself is positive definite: \[\begin{alignedat}{4} \langle x, x\rangle > 0 & \quad \text{ if } x \neq 0, \\ \langle x, x\rangle = 0 & \quad \text{ if } x = 0\,. \end{alignedat}\]

It follows from properties 1 and 2 that a complex inner product is antilinear, also called conjugate linear, in its second argument, meaning that \[\langle x, ay_1 + by_2\rangle = \bar{a}\langle x, y_1\rangle + \bar{b}\langle x, y_2\rangle\,.\]

A real inner product space is defined in the same way, except that H is a real vector space and the inner product takes real values. Such an inner product will be a bilinear map and \((H, H, \langle \cdot, \cdot \rangle)\) will form a dual system.

The norm is the real-valued function \[\|x\| = \sqrt{\langle x, x \rangle}\,,\] and the distance \(d\) between two points \(x, y\) in H is defined in terms of the norm by \[d(x, y) = \|x - y\| = \sqrt{\langle x - y, x - y \rangle}\,.\] Here, \(d(x,y)\) is a distance function meaning firstly that it is symmetric in \(x\) and \(y,\) secondly that the distance between \(x\) and itself is zero, and otherwise the distance between \(x\) and \(y\) must be positive, and lastly that the triangle inequality holds, meaning that the length of one leg of a triangle xyz cannot exceed the sum of the lengths of the other two legs: \[d(x, z) \leq d(x, y) + d(y, z)\,.\]

This last property is ultimately a consequence of the more fundamental Cauchy-Schwarz inequality, which asserts \[\left|\langle x, y\rangle\right| \leq \|x\| \|y\|\] with equality if and only if \(x\) and \(y\) are linearly dependent.

With a distance function defined in this way, any inner product space is a metric space. An inner product space is sometimes known as a pre-Hilbert space. Any pre-Hilbert space that is additionally also a complete space is a Hilbert space.

Condensed: the full section is in Wikipedia.

The sequence space \(\ell^2\) consists of all infinite sequences z = (z₁, z₂, ...) of complex numbers such that the series \[\sum_{n=1}^\infty |z_n|^2\] of its squared norms converges. The inner product on \(\ell^2\) is defined by \[\langle \mathbf{z}, \mathbf{w}\rangle = \sum_{n=1}^\infty z_n \overline{w}_n \, .\] The series for the inner product converges as a consequence of the Cauchy-Schwarz inequality and the assumed convergence of the two series of squared norms.

Completeness of the space holds provided that whenever a series of elements from \(\ell^2\) converges absolutely (in norm), then it converges to an element of \(\ell^2\). The proof is basic in mathematical analysis, and permits mathematical series of elements of the space to be manipulated with the same ease as series of complex numbers (or vectors in a finite-dimensional Euclidean space).

Prior to the development of Hilbert spaces, other generalizations of Euclidean spaces were known to mathematicians and physicists. In particular, the idea of an abstract linear space (vector space) had gained some traction towards the end of the 19th century: this is a space whose elements can be added together and multiplied by scalars (such as real or complex numbers) without necessarily identifying these elements with "geometric" vectors, such as position and momentum vectors in physical systems. Other objects studied at the turn of the 20th century, in particular spaces of sequences (including series) and spaces of functions, can naturally be thought of as linear spaces. Functions, for instance, can be added together or multiplied by constant scalars, and these operations obey the algebraic laws satisfied by addition and scalar multiplication of spatial vectors.

In the first decade of the 20th century, parallel developments led to the introduction of Hilbert spaces. The first of these was the observation, which arose during David Hilbert and Erhard Schmidt's study of integral equations, that two square-integrable real-valued functions f and g on an interval [a, b] have an inner product

\(\langle f, g \rangle = \int_a^b f(x)g(x)\, \mathrm{d}x\)

that has many of the familiar properties of the Euclidean dot product. In particular, the idea of an orthogonal family of functions has meaning. Schmidt exploited the similarity of this inner product with the usual dot product to prove an analog of the spectral decomposition for an operator of the form

\(f(x) \mapsto \int_a^b K(x, y) f(y)\, \mathrm{d}y\)

where K is a continuous function symmetric in x and y. The resulting eigenfunction expansion expresses the function K as a series of the form

\(K(x, y) = \sum_n \lambda_n\varphi_n(x)\varphi_n(y)\)

where the functions φ_n are orthogonal in the sense that ⟨φ_n, φ_m⟩ = 0 for all n ≠ m. The individual terms in this series are sometimes referred to as elementary product solutions. However, there are eigenfunction expansions that fail to converge in a suitable sense to a square-integrable function: the missing ingredient, which ensures convergence, is completeness.

The second development was the Lebesgue integral, an alternative to the Riemann integral introduced by Henri Lebesgue in 1904. The Lebesgue integral made it possible to integrate a much broader class of functions. In 1907, Frigyes Riesz and Ernst Sigismund Fischer independently proved that the space L of square Lebesgue-integrable functions is a complete metric space. As a consequence of the interplay between geometry and completeness, the 19th century results of Joseph Fourier, Friedrich Bessel and Marc-Antoine Parseval on trigonometric series easily carried over to these more general spaces, resulting in a geometrical and analytical apparatus now usually known as the Riesz-Fischer theorem.

Condensed: the full section is in Wikipedia.

Lebesgue spaces are function spaces associated to measure spaces (X, M, μ), where X is a set, M is a σ-algebra of subsets of X, and μ is a countably additive measure on M. Let L(X, μ) be the space of those complex-valued measurable functions on X for which the Lebesgue integral of the square of the absolute value of the function is finite, i.e., for a function f in L(X, μ), \[\int_X |f|^2 \, \mathrm{d} \mu < \infty \,,\] and where functions are identified if and only if they differ only on a set of measure zero.

The inner product of functions f and g in L(X, μ) is then defined as \[\langle f, g\rangle = \int_X f(t) \overline{g(t)} \, \mathrm{d} \mu(t)\] or \[\langle f, g\rangle = \int_X \overline{f(t)} g(t) \, \mathrm{d} \mu(t) \,,\]

where the second form (conjugation of the first element) is commonly found in the theoretical physics literature. For f and g in L, the integral exists because of the Cauchy-Schwarz inequality, and defines an inner product on the space. Equipped with this inner product, L is in fact complete. The Lebesgue integral is essential to ensure completeness: on domains of real numbers, for instance, not enough functions are Riemann integrable.

The Lebesgue spaces appear in many natural settings. The spaces L(R) and L([0,1]) of square-integrable functions with respect to the Lebesgue measure on the real line and unit interval, respectively, are natural domains on which to define the Fourier transform and Fourier series. In other situations, the measure may be something other than the ordinary Lebesgue measure on the real line. For instance, if w is any positive measurable function, the space of all measurable functions f on the interval [0, 1] satisfying \[\int_0^1 \bigl|f(t)\bigr|^2 w(t)\, \mathrm{d}t < \infty\] is called the weighted L space L
_w([0, 1]), and w is called the weight function. The inner product is defined by \[\langle f, g\rangle = \int_0^1 f(t) \overline{g(t)} w(t) \, \mathrm{d}t \,.\]

The weighted space L
_w([0, 1]) is identical with the Hilbert space L([0, 1], μ) where the measure μ of a Lebesgue-measurable set A is defined by \[\mu(A) = \int_A w(t)\,\mathrm{d}t \,.\]

Weighted L spaces like this are frequently used to study orthogonal polynomials, because different families of orthogonal polynomials are orthogonal with respect to different weighting functions.

Sobolev spaces, denoted by H or W, are Hilbert spaces. These are a special kind of function space in which differentiation may be performed, but that (unlike other Banach spaces such as the Hölder spaces) support the structure of an inner product. Because differentiation is permitted, Sobolev spaces are a convenient setting for the theory of partial differential equations. They also form the basis of the theory of direct methods in the calculus of variations.

For s a non-negative integer and Ω ⊂ R, the Sobolev space H(Ω) contains L functions whose weak derivatives of order up to s are also L. The inner product in H(Ω) is \[\langle f, g\rangle = \int_\Omega f(x)\bar{g}(x)\,\mathrm{d}x + \int_\Omega D f(x)\cdot D\bar{g}(x)\,\mathrm{d}x + \cdots + \int_\Omega D^s f(x)\cdot D^s \bar{g}(x)\, \mathrm{d}x\] where the dot indicates the dot product in the Euclidean space of partial derivatives of each order. Sobolev spaces can also be defined when s is not an integer.

Sobolev spaces are also studied from the point of view of spectral theory, relying more specifically on the Hilbert space structure. If Ω is a suitable domain, then one can define the Sobolev space H(Ω) as the space of Bessel potentials; roughly, \[H^s(\Omega) = \left\{ (1-\Delta)^{-s/2}f \mathrel{\Big|} f\in L^2(\Omega)\right\} \,.\]

Here Δ is the Laplacian and (1 − Δ) is understood in terms of the spectral mapping theorem. Apart from providing a workable definition of Sobolev spaces for non-integer s, this definition also has particularly desirable properties under the Fourier transform that make it ideal for the study of pseudodifferential operators. Using these methods on a compact Riemannian manifold, one can obtain for instance the Hodge decomposition, which is the basis of Hodge theory.

Many of the applications of Hilbert spaces exploit the fact that Hilbert spaces support generalizations of simple geometric concepts like projection and change of basis from their usual finite dimensional setting. In particular, the spectral theory of continuous self-adjoint linear operators on a Hilbert space generalizes the usual spectral decomposition of a matrix, and this often plays a major role in applications of the theory to other areas of mathematics and physics.

In the theory of ordinary differential equations, spectral methods on a suitable Hilbert space are used to study the behavior of eigenvalues and eigenfunctions of differential equations. For example, the Sturm-Liouville problem arises in the study of the harmonics of waves in a violin string or a drum, and is a central problem in ordinary differential equations. The problem is a differential equation of the form \[-\frac{\mathrm{d}}{\mathrm{d}x}\left[p(x)\frac{\mathrm{d}y}{\mathrm{d}x}\right] + q(x)y = \lambda w(x)y\] for an unknown function y on an interval [a, b], satisfying general homogeneous Robin boundary conditions \[\begin{cases} \alpha y(a)+\alpha' y'(a) &= 0 \\ \beta y(b) + \beta' y'(b) &= 0 \,. \end{cases}\] The functions p, q, and w are given in advance, and the problem is to find the function y and constants λ for which the equation has a solution. The problem only has solutions for certain values of λ, called eigenvalues of the system, and this is a consequence of the spectral theorem for compact operators applied to the integral operator defined by the Green's function for the system. Furthermore, another consequence of this general result is that the eigenvalues λ of the system can be arranged in an increasing sequence tending to infinity.

Hilbert spaces form a basic tool in the study of partial differential equations. For many classes of partial differential equations, such as linear elliptic equations, it is possible to consider a generalized solution (known as a weak solution) by enlarging the class of functions. Many weak formulations involve the class of Sobolev functions, which is a Hilbert space. A suitable weak formulation reduces to a geometrical problem, the analytic problem of finding a solution or, often what is more important, showing that a solution exists and is unique for given boundary data. For linear elliptic equations, one geometrical result that ensures unique solvability for a large class of problems is the Lax-Milgram theorem. This strategy forms the rudiment of the Galerkin method (a finite element method) for numerical solution of partial differential equations.

An example is the Poisson equation −Δu = g with Dirichlet boundary conditions in a bounded domain Ω in R. The weak formulation consists of finding a function u such that, for all continuously differentiable functions v in Ω vanishing on the boundary: \[\int_\Omega \nabla u\cdot\nabla v = \int_\Omega gv\,.\]

This can be recast in terms of the Hilbert space H
₀(Ω) consisting of functions u such that u, along with its weak partial derivatives, are square integrable on Ω, and vanish on the boundary. The question then reduces to finding u in this space such that for all v in this space \[a(u, v) = b(v)\]

where a is a continuous bilinear form, and b is a continuous linear functional, given respectively by \[a(u, v) = \int_\Omega \nabla u\cdot\nabla v,\quad b(v)= \int_\Omega gv\,.\]

Since the Poisson equation is elliptic, it follows from Poincaré's inequality that the bilinear form a is coercive. The Lax-Milgram theorem then ensures the existence and uniqueness of solutions of this equation.

Hilbert spaces allow for many elliptic partial differential equations to be formulated in a similar way, and the Lax-Milgram theorem is then a basic tool in their analysis. With suitable modifications, similar techniques can be applied to parabolic partial differential equations and certain hyperbolic partial differential equations.

The field of ergodic theory is the study of the long-term behavior of chaotic dynamical systems. The protypical case of a field that ergodic theory applies to is thermodynamics, in which, though the microscopic state of a system is extremely complicated (it is impossible to understand the ensemble of individual collisions between particles of matter), the average behavior over sufficiently long time intervals is tractable. The laws of thermodynamics are assertions about such average behavior. In particular, one formulation of the zeroth law of thermodynamics asserts that over sufficiently long timescales, the only functionally independent measurement that one can make of a thermodynamic system in equilibrium is its total energy, in the form of temperature.

An ergodic dynamical system is one for which, apart from the energy, measured by the Hamiltonian, there are no other functionally independent conserved quantities on the phase space. More explicitly, suppose that the energy E is fixed, and let Ω_E be the subset of the phase space consisting of all states of energy E (an energy surface), and let T_t denote the evolution operator on the phase space. The dynamical system is ergodic if every invariant measurable function on Ω_E is constant almost everywhere. An invariant function f is one for which \[f(T_tw) = f(w)\] for all w on Ω_E and all time t. Liouville's theorem implies that there exists a measure μ on the energy surface that is invariant under the time translation. As a result, time translation is a unitary transformation of the Hilbert space L(Ω_E, μ) consisting of square-integrable functions on the energy surface Ω_E with respect to the inner product \[\left\langle f, g\right\rangle_{L^2\left(\Omega_E, \mu\right)} = \int_{\Omega_E} f\bar{g}\,\mathrm{d}\mu\,.\]

The von Neumann mean ergodic theorem states the following:

• If U_t is a (strongly continuous) one-parameter semigroup of unitary operators on a Hilbert space H, and P is the orthogonal projection onto the space of common fixed points of U_t, {x ∈H | U_tx = x, ∀t > 0}, then \[Px = \lim_{T\to\infty} \frac{1}{T} \int_0^T U_tx\,\mathrm{d}t\,.\]

For an ergodic system, the fixed set of the time evolution consists only of the constant functions, so the ergodic theorem implies the following: for any function f ∈ L(Ω_E, μ), \[\underset{T\to\infty}{L^2 - \lim} \frac{1}{T}\int_0^T f(T_tw)\,\mathrm{d}t = \int_{\Omega_E} f(y)\,\mathrm{d}\mu(y)\,.\]

That is, the long time average of an observable f is equal to its expectation value over an energy surface.

One of the basic goals of Fourier analysis is to decompose a function into a (possibly infinite) linear combination of given basis functions: the associated Fourier series. The classical Fourier series associated to a function f defined on the interval [0, 1] is a series of the form \[\sum_{n=-\infty}^\infty a_n e^{2\pi in\theta}\] where \[a_n = \int_0^1f(\theta)\;\!e^{-2\pi in\theta}\,\mathrm{d}\theta\,.\]

The example of adding up the first few terms in a Fourier series for a sawtooth function is shown in the figure. The basis functions are sine waves with wavelengths ⁠λ/n⁠ (for integer n) shorter than the wavelength λ of the sawtooth itself (except for n = 1, the fundamental wave).

A significant problem in classical Fourier series asks in what sense the Fourier series converges, if at all, to the function f. Hilbert space methods provide one possible answer to this question. The functions e_n(θ) = e form an orthogonal basis of the Hilbert space L([0, 1]). Consequently, any square-integrable function can be expressed as a series \[f(\theta) = \sum_n a_n e_n(\theta)\,,\quad a_n = \langle f, e_n\rangle\] and, moreover, this series converges in the Hilbert space sense (that is, in the L mean).

The problem can also be studied from the abstract point of view: every Hilbert space has an orthonormal basis, and every element of the Hilbert space can be written in a unique way as a sum of multiples of these basis elements. The coefficients appearing on these basis elements are sometimes known abstractly as the Fourier coefficients of the element of the space. The abstraction is especially useful when it is more natural to use different basis functions for a space such as L([0, 1]). In many circumstances, it is desirable not to decompose a function into trigonometric functions, but rather into orthogonal polynomials or wavelets for instance, and in higher dimensions into spherical harmonics.

For instance, if e_n are any orthonormal basis functions of L[0, 1], then a given function in L[0, 1] can be approximated as a finite linear combination \[f(x) \approx f_n (x) = a_1 e_1 (x) + a_2 e_2(x) + \cdots + a_n e_n (x)\,.\]

The coefficients {a_j} are selected to make the magnitude of the difference ‖f − f_n‖ as small as possible. Geometrically, the best approximation is the orthogonal projection of f onto the subspace consisting of all linear combinations of the {e_j}, and can be calculated by \[a_j = \int_0^1 \overline{e_j(x)}f (x) \, \mathrm{d}x\,.\]

That this formula minimizes the difference ‖f − f_n‖ is a consequence of Bessel's inequality and Parseval's formula.

Condensed: the full section is in Wikipedia.

In the mathematically rigorous formulation of quantum mechanics, developed by John von Neumann, the possible states (more precisely, the pure states) of a quantum mechanical system are represented by unit vectors (called state vectors) residing in a complex separable Hilbert space, known as the state space, well defined up to a complex number of norm 1 (the phase factor). In other words, the possible states are points in the projectivization of a Hilbert space, usually called the complex projective space. The exact nature of this Hilbert space is dependent on the system; for example, the position and momentum states for a single non-relativistic spin zero particle is the space of all square-integrable functions, while the states for the spin of a single proton are unit elements of the two-dimensional complex Hilbert space of spinors. Each observable is represented by a self-adjoint linear operator acting on the state space. Each eigenstate of an observable corresponds to an eigenvector of the operator, and the associated eigenvalue corresponds to the value of the observable in that eigenstate.

The inner product between two state vectors is a complex number known as a probability amplitude. During an ideal measurement of a quantum mechanical system, the probability that a system collapses from a given initial state to a particular eigenstate is given by the square of the absolute value of the probability amplitudes between the initial and final states. The possible results of a measurement are the eigenvalues of the operator, which explains the choice of self-adjoint operators, for all the eigenvalues must be real. The probability distribution of an observable in a given state can be found by computing the spectral decomposition of the corresponding operator.

For a general system, states are typically not pure, but instead are represented as statistical mixtures of pure states, or mixed states, given by density matrices: self-adjoint operators of trace one on a Hilbert space. Moreover, for general quantum mechanical systems, the effects of a single measurement can influence other parts of a system in a manner that is described instead by a positive operator valued measure. Thus the structure both of the states and observables in the general theory is considerably more complicated than the idealization for pure states.

In probability theory, Hilbert spaces also have diverse applications. Here a fundamental Hilbert space is the space of random variables on a given probability space, having class \(L^2\) (finite first and second moments). A common operation in statistics is that of centering a random variable by subtracting its expectation. Thus if \(X\) is a random variable, then \(X - E(X)\) is its centering. In the Hilbert space view, this is the orthogonal projection of \(X\) onto the kernel of the expectation operator, which a continuous linear functional on the Hilbert space (in fact, the inner product with the constant random variable 1), and so this kernel is a closed subspace.

The conditional expectation has a natural interpretation in the Hilbert space. Suppose that a probability space \((\Omega, P, \mathcal B)\) is given, where \(\mathcal B\) is a sigma algebra on the set \(\Omega\), and \(P\) is a probability measure on the measure space \((\Omega, \mathcal B)\). If \(\mathcal F\le\mathcal B\) is a sigma subalgebra of \(\mathcal B\), then the conditional expectation \(E[X\mid\mathcal F]\) is the orthogonal projection of \(X\) onto the subspace of \(L^2(\Omega, P)\) consisting of the \(\mathcal F\)-measurable functions. If the random variable \(X\) in \(L^2(\Omega, P)\) is independent of the sigma algebra \(\mathcal F\) then conditional expectation \(E(X\mid\mathcal F) = E(X)\), i.e., its projection onto the \(\mathcal F\)-measurable functions is constant. Equivalently, the projection of its centering is zero.

In particular, if two random variables \(X\) and \(Y\) (in \(L^2(\Omega, P)\)) are independent, then the centered random variables \(X-E(X)\) and \(Y-E(Y)\) are orthogonal. (This means that the two variables have zero covariance: they are uncorrelated.) In that case, the Pythagorean theorem in the kernel of the expectation operator implies that the variances of \(X\) and \(Y\) satisfy the identity: \[\operatorname{Var}(X+Y) = \operatorname{Var}(X) + \operatorname{Var}(Y),\] sometimes called the Pythagorean theorem of statistics, and is of importance in linear regression. The analysis of variance could use the Pythagorean Theorem so that the variance is viewed as the decomposition of the squared length of a vector into the sum of the squared lengths of several vectors.

The theory of martingales can be formulated in Hilbert spaces. A martingale in a Hilbert space is a sequence \(x_1,x_2,\dots\) of elements of a Hilbert space such that, for each n, \(x_n\) is the orthogonal projection of \(x_{n+1}\) onto the linear hull of \(x_1,\dots,x_n\). If the \(x_k\) are random variables, this reproduces the usual definition of a (discrete) martingale: the expectation of \(x_{n+1}\), conditioned on \(x_1,\dots,x_n\), is equal to \(x_n\).

Condensed: the full section is in Wikipedia.