Sequences of functions and uniform convergence

In the mathematical field of analysis, uniform convergence is a mode of convergence of functions stronger than pointwise convergence. A sequence of functions \((f_n)\) converges uniformly to a limiting function \(f\) if, roughly speaking, they uniformly approximate the function \(f\) over the whole domain, meaning that all but finitely many of the functions of the sequence lie in a uniform error bar of the original function. Graphically this means that, given any thin band around the graph of \(f\), the graphs of all but finitely many of the functions \(f_n\) lie within that thin band. This is in contrast to pointwise convergence, in which all but finitely many of the functions lie in a thin band at each point, but the finite set of functions which must be excluded in order for that to be the case varies from point to point.

The strength of uniform convergence makes it ideal in many applications, where pointwise convergence is not sufficient. For example, the uniform limit of a sequence of continuous functions is automatically continuous; the uniform limit of Riemann integrable functions is automatically Riemann integrable. With additional hypotheses, differentiability can be transferred to the limit function as well. The difference between uniform convergence and pointwise convergence was not fully appreciated early in the history of calculus, leading to instances of faulty reasoning. The concept was first formalized by Karl Weierstrass.

In 1821 Augustin-Louis Cauchy published a proof that a convergent sum of continuous functions is always continuous, to which Niels Henrik Abel in 1826 found purported counterexamples in the context of Fourier series, arguing that Cauchy's proof had to be incorrect. Completely standard notions of convergence did not exist at the time, and Cauchy handled convergence using infinitesimal methods. When put into the modern language, what Cauchy proved is that a uniformly convergent sequence of continuous functions has a continuous limit. The failure of a merely pointwise-convergent limit of continuous functions to converge to a continuous function illustrates the importance of distinguishing between different types of convergence when handling sequences of functions.

The term uniform convergence was probably first used by Christoph Gudermann, in an 1838 paper on elliptic functions, where he employed the phrase "convergence in a uniform way" when the "mode of convergence" of a series \(\sum_{n=1}^\infty f_n(x,\phi,\psi)\) is independent of the variables \(\phi\) and \(\psi.\) While he thought it a "remarkable fact" when a series converged in this way, he did not give a formal definition, nor use the property in any of his proofs.

Later Gudermann's pupil, Karl Weierstrass, who attended his course on elliptic functions in 1839-1840, coined the term gleichmäßig konvergent (German: uniformly convergent) which he used in his 1841 paper Zur Theorie der Potenzreihen, published in 1894. Independently, similar concepts were articulated by Philipp Ludwig von Seidel and George Gabriel Stokes. G. H. Hardy compares the three definitions in his paper "Sir George Stokes and the concept of uniform convergence" and remarks: "Weierstrass's discovery was the earliest, and he alone fully realized its far-reaching importance as one of the fundamental ideas of analysis."

Under the influence of Weierstrass and Bernhard Riemann this concept and related questions were intensely studied at the end of the 19th century by Hermann Hankel, Paul du Bois-Reymond, Ulisse Dini, Cesare Arzelà and others.

We first define uniform convergence for real-valued functions, although the concept is readily generalized to functions mapping to metric spaces and, more generally, uniform spaces (see below).

Suppose \(E\) is a set and \((f_n)_{n \in \N}\) is a sequence of real-valued functions defined on it. We say the sequence \((f_n)_{n \in \N}\) is uniformly convergent on \(E\) with limit \(f: E \to \R\) if for every \(\varepsilon > 0,\) there exists a natural number \(N\) such that for all \(n \geq N\) and for all \(x \in E\)

\(\bigl|f_n(x)-f(x)\bigr|<\varepsilon.\)

The notation for uniform convergence of \(f_n\) to \(f\) is not quite standardized and different authors have used a variety of symbols, including (in roughly decreasing order of popularity):

\(f_n\rightrightarrows f, \quad \underset{n\to\infty}{\mathrm{unif\ lim}}f_n = f, \quad f_n \overset{\mathrm{unif.}}{\longrightarrow} f, \quad f=\mathrm{u}\!\!-\!\!\!\lim_{n\to\infty} f_n .\)

Frequently, no special symbol is used, and authors simply write

\(f_n\to f \quad \mathrm{uniformly}\)

to indicate that convergence is uniform. (In contrast, the expression \(f_n\to f\) on \(E\) without an adverb is taken to mean pointwise convergence on \(E\): for all \(x \in E\), \(f_n(x)\to f(x)\) as \(n\to\infty\).)

Since \(\R\) is a complete metric space, the Cauchy criterion can be used to give an equivalent alternative formulation for uniform convergence: \((f_n)_{n\in\N}\) converges uniformly on \(E\) (in the previous sense) if and only if for every \(\varepsilon > 0\), there exists a natural number \(N\) such that

\(x\in E, m,n\geq N \implies \bigl|f_m(x)-f_n(x)\bigr|<\varepsilon\).

In yet another equivalent formulation (derived by using the definition of limit and supremum), if we define

\(d_n = \sup_{x\in E} \bigl|f_n(x) - f(x)\bigr|,\)

\(d(f,g)=\sup_{x\in E} \bigl|f(x)-g(x)\bigr|.\)

\(f_n\rightrightarrows f\iff d(f_n,f) \to 0\).

Condensed: the full section is in Wikipedia.

One may straightforwardly extend the concept to functions E → M, where (M, d) is a metric space, by replacing \(|f_n(x)-f(x)|\) with \(d(f_n(x),f(x))\).

The most general setting is the uniform convergence of nets of functions E → X, where X is a uniform space. We say that the net \((f_\alpha)\) converges uniformly with limit f : E → X if and only if for every entourage V in X, there exists an \(\alpha_0\), such that for every x in E and every \(\alpha\geq \alpha_0\), \((f_\alpha(x),f(x))\) is in V. In this situation, uniform limit of continuous functions remains continuous.

Uniform convergence admits a simplified definition in a hyperreal setting. Thus, a sequence \(f_n\) converges to f uniformly if for all hyperreal x in the domain of \({f}^*\) and all infinite n, \({f}_n^*(x)\) is infinitely close to \({f}^*(x)\) (see microcontinuity for a similar definition of uniform continuity). In contrast, pointwise continuity requires this only for real x.

For \(x \in [0,1)\), a basic example of uniform convergence can be illustrated as follows: the sequence \(\textstyle \bigl(\tfrac12\bigr)\vphantom)^{x+n}\) converges uniformly, while \(x^n\) does not. Specifically, assume \(\varepsilon=\tfrac14\). Each function \(\textstyle \bigl(\tfrac12\bigr)\vphantom)^{x+n}\) is less than or equal to \(\tfrac14\) when \(n \geq 2\), regardless of the value of \(x\). On the other hand, \(x^n\) is only less than or equal to \(\tfrac14\) at ever increasing values of \(n\) when values of \(x\) are selected closer and closer to 1 (explained more in depth further below).

Given a topological space X, we can equip the space of bounded real or complex-valued functions over X with the uniform norm topology, with the uniform metric defined by

\(d(f,g)={\|f-g\|}_{\infty}=\sup_{x\in X} \bigl|f(x)-g(x)\bigr|.\)

Then uniform convergence simply means convergence in the uniform norm topology:

\(\lim_{n\to\infty}{\|f_n-f\|}_{\infty}=0\).

The sequence of functions \((f_n)\)

\(\begin{cases} f_n:[0,1]\to [0,1] \\[3mu] f_n(x)=x^n \end{cases}\)

is a classic example of a sequence of functions that converges to a function \(f\) pointwise but not uniformly. To show this, we first observe that the pointwise limit of \((f_n)\) as \(n\to\infty\) is the function \(f\), given by

\(f(x) = \lim_{n\to \infty} f_n(x) = \begin{cases} 0, & x \in [0,1); \\[3mu] 1, & x=1. \end{cases}\)

Pointwise convergence: Convergence is trivial for \(x=0\) and \(x=1\), since \(f_n(0)=f(0)=0\) and \(f_n(1)=f(1)=1\), for all \(n\). For \(x \in (0,1)\) and given \(\varepsilon>0\), we can ensure that \(|f_n(x)-f(x)|<\varepsilon\) whenever \(n\geq N\) by choosing \(N = \lceil\log\varepsilon/\log x\rceil\), which is the minimum integer exponent of \(x\) that allows it to reach or dip below \(\varepsilon\) (here the upper square brackets indicate rounding up, see ceiling function). Hence, \(f_n\to f\) pointwise for all \(x\in[0,1]\). Note that the choice of \(N\) depends on the value of \(\varepsilon\) and \(x\). Moreover, for a fixed choice of \(\varepsilon\), \(N\) (which cannot be defined to be smaller) grows without bound as \(x\) approaches 1. These observations preclude the possibility of uniform convergence.

Non-uniformity of convergence: The convergence is not uniform, because we can find an \(\varepsilon>0\) so that no matter how large we choose \(N,\) there will be values of \(x \in [0,1]\) and \(n \geq N\) such that \(|f_n(x)-f(x)|\geq\varepsilon.\) To see this, first observe that regardless of how large \(n\) becomes, there is always an \(x_0 \in [0,1)\) such that \(f_n(x_0)=\tfrac12.\) Thus, if we choose \(\varepsilon = \tfrac14,\) we can never find an \(N\) such that \(|f_n(x)-f(x)|<\varepsilon\) for all \(x\in[0,1]\) and \(n\geq N\). Explicitly, whatever candidate we choose for \(N\), consider the value of \(f_N\) at \(\textstyle x_0 = \bigl(\tfrac12\bigr)\vphantom)^{1/N}\). Since

\(\bigl|f_N(x_0) - f(x_0)\bigr| = \left| {\Bigl( \bigl(\tfrac{1}{2} \bigr)^{1/N} \Bigr)\vphantom{\big|}^N\! - 0}~\! \right| = \tfrac{1}{2} > \tfrac{1}{4} = \varepsilon,\)

\(\lim_{n\to\infty}\|f_n-f\|_{\infty}=1,\)

Condensed: the full section is in Wikipedia.

The series expansion of the exponential function can be shown to be uniformly convergent on any bounded subset \(S \subset \C\) using the Weierstrass M-test.

Theorem (Weierstrass M-test). Let \((f_n)\) be a sequence of functions \(f_n:E\to \C\) and let \(M_n\) be a sequence of positive real numbers such that \(|f_n(x)|\le M_n\) for all \(x\in E\) and \(n=1,2, 3, \ldots\) If \(\sum_n M_n\) converges, then \(\sum_n f_n\) converges absolutely and uniformly on \(E\).

The complex exponential function can be expressed as the series:

\(\sum_{n=0}^{\infty}\frac{z^n}{n!}.\)

Any bounded subset is a subset of some disc \(D_R\) of radius \(R,\) centered on the origin in the complex plane. The Weierstrass M-test requires us to find an upper bound \(M_n\) on the terms of the series, with \(M_n\) independent of the position in the disc:

\(\left| \frac{z^n}{n!} \right|\le M_n, \forall z\in D_R.\)

To do this, we notice

\(\left| \frac{z^n}{n!}\right| \le \frac{|z|^n}{n!} \le \frac{R^n}{n!}\)

and take \(M_n=\tfrac{R^n}{n!}.\)

If \(\sum_{n=0}^{\infty}M_n\) is convergent, then the M-test asserts that the original series is uniformly convergent.

\(\lim_{n \to \infty}\frac{M_{n+1}}{M_n}=\lim_{n \to \infty}\frac{R^{n+1}}{R^n}\frac{n!}{(n+1)!}=\lim_{n \to \infty}\frac{R}{n+1}=0\)

Condensed: the full section is in Wikipedia.

• Every uniformly convergent sequence is locally uniformly convergent.
• Every locally uniformly convergent sequence is compactly convergent.
• For locally compact spaces local uniform convergence and compact convergence coincide.
• A sequence of continuous functions on metric spaces, with the image metric space being complete, is uniformly convergent if and only if it is uniformly Cauchy.
• If \(S\) is a compact interval (or in general a compact topological space), and \((f_n)\) is a monotone increasing sequence (meaning \(f_n(x) \leq f_{n+1}(x)\) for all n and x) of continuous functions with a pointwise limit \(f\) which is also continuous, then the convergence is necessarily uniform (Dini's theorem). Uniform convergence is also guaranteed if \(S\) is a compact interval and \((f_n)\) is an equicontinuous sequence that converges pointwise.

If \(E\) and \(M\) are topological spaces, then it makes sense to talk about the continuity of the functions \(f_n,f:E\to M\). If we further assume that \(M\) is a metric space, then (uniform) convergence of the \(f_n\) to \(f\) is also well defined. The following result states that continuity is preserved by uniform convergence:

Uniform limit theorem, Suppose \(E\) is a topological space, \(M\) is a metric space, and \((f_n)\) is a sequence of continuous functions \(f_n:E\to M\). If \(f_n \rightrightarrows f\) on \(E\), then \(f\) is also continuous.

This theorem is proved by the "⁠\(\varepsilon/3\)⁠ trick", and is the archetypal example of this trick: to prove a given inequality (that a desired quantity is less than ⁠\(\varepsilon\)⁠), one uses the definitions of continuity and uniform convergence to produce 3 inequalities (demonstrating three separate quantities are each less than ⁠\(\varepsilon/3\)⁠), and then combines them via the triangle inequality to produce the desired inequality.

Let \(x_0\in E\) be an arbitrary point. We will prove that \(f\) is continuous at \(x_0\). Let \(\varepsilon >0\). By uniform convergence, there exists a natural number \(N\) such that

\(\forall x \in E\quad d\bigl(f_N(x),f(x)\bigr)\leq \tfrac13\varepsilon\)

(uniform convergence shows that the above statement is true for all \(n\geq N\), but we will only use it for one function of the sequence, namely \(f_N\)).

It follows from the continuity of \(f_N\) at \(x_0\in E\) that there exists an open set \(U\) containing \(x_0\) such that

\(\forall x\in U\quad d\bigl(f_N(x),f_N(x_0)\bigr)\leq\tfrac13\varepsilon\).

Hence, using the triangle inequality,

\(\forall x \in U\quad d\bigl(f(x), f(x_0)\bigr) \leq d\bigl(f(x),f_N(x)\bigr) + d\bigl(f_N(x),f_N(x_0)\bigr) + d\bigl(f_N(x_0),f(x_0)\bigr) \leq \varepsilon\),

which gives us the continuity of \(f\) at \(x_0\).\(\quad\square\)

This theorem is an important one in the history of real and Fourier analysis, since many 18th century mathematicians had the intuitive understanding that a sequence of continuous functions always converges to a continuous function. The image above shows a counterexample, and many discontinuous functions could, in fact, be written as a Fourier series of continuous functions. The erroneous claim that the pointwise limit of a sequence of continuous functions is continuous (originally stated in terms of convergent series of continuous functions) is infamously known as "Cauchy's wrong theorem". The uniform limit theorem shows that a stronger form of convergence, uniform convergence, is needed to ensure the preservation of continuity in the limit function.

More precisely, this theorem states that the uniform limit of uniformly continuous functions is uniformly continuous; for a locally compact space, continuity is equivalent to local uniform continuity, and thus the uniform limit of continuous functions is continuous.

If \(S\) is an interval and all the functions \(f_n\) are differentiable and converge to a limit \(f\), it is often desirable to determine the derivative function \(f'\) by taking the limit of the sequence \(f'_n\). This is however in general not possible: even if the convergence is uniform, the limit function need not be differentiable (not even if the sequence consists of everywhere-analytic functions, see Weierstrass function), and even if it is differentiable, the derivative of the limit function need not be equal to the limit of the derivatives. Consider for instance \(f_n(x) = n^{-1/2}{\sin(nx)}\) with uniform limit \(f_n\rightrightarrows f\equiv 0\). Clearly, \(f'\) is also identically zero. However, the derivatives of the sequence of functions are given by \(f'_n(x)=n^{1/2}\cos nx,\) and the sequence \(f'_n\) does not converge to \(f',\) or even to any function at all. In order to ensure a connection between the limit of a sequence of differentiable functions and the limit of the sequence of derivatives, the uniform convergence of the sequence of derivatives plus the convergence of the sequence of functions at at least one point is required:

If \((f_n)\) is a sequence of differentiable functions on \([a,b]\) such that \(\lim_{n\to\infty} f_n(x_0)\) exists (and is finite) for some \(x_0\in[a,b]\) and the sequence \((f'_n)\) converges uniformly on \([a,b]\), then \(f_n\) converges uniformly to a function \(f\) on \([a,b]\), and \(f'(x) = \lim_{n\to \infty} f'_n(x)\) for \(x \in [a, b]\).

Similarly, one often wants to exchange integrals and limit processes. For the Riemann integral, this can be done if uniform convergence is assumed:

If \({(f_n)}_{n=1}^\infty\) is a sequence of Riemann integrable functions defined on a compact interval \(I\) which uniformly converge with limit \(f\), then \(f\) is Riemann integrable and its integral can be computed as the limit of the integrals of the \(f_n\): \[\int_I f = \lim_{n\to\infty}\int_I f_n.\]

In fact, for a uniformly convergent family of bounded functions on an interval, the upper and lower Riemann integrals converge to the upper and lower Riemann integrals of the limit function. This follows because, for n sufficiently large, the graph of \(f_n\) is within ε of the graph of f, and so the upper sum and lower sum of \(f_n\) are each within \(\varepsilon |I|\) of the value of the upper and lower sums of \(f\), respectively.

Much stronger theorems in this respect, which require not much more than pointwise convergence, can be obtained if one abandons the Riemann integral and uses the Lebesgue integral instead.

Using Morera's Theorem, one can show that if a sequence of analytic functions converges uniformly in a region S of the complex plane, then the limit is analytic in S. This example demonstrates that complex functions are more well-behaved than real functions, since the uniform limit of analytic functions on a real interval need not even be differentiable (see Weierstrass function).

We say that \(\sum_{n=1}^\infty f_n\) converges:

1. pointwise on E if and only if the sequence of partial sums \(\textstyle s_n(x)=\sum_{j=1}^{n} f_j(x)\) converges for every \(x\in E\).
2. uniformly on E if and only if s_n converges uniformly as \(n\to\infty\).
3. absolutely on E if and only if \(\sum_{n=1}^\infty |f_n|\) converges for every \(x \in E\).

With this definition comes the following result:

Uniform convergence is the norm topology on several important function spaces. For example, let \(C([0,1])\) denote the space of real-valued or complex-valued continuous functions on the unit interval. With the supremum norm \[\|f\|_\infty=\sup_{x\in[0,1]} |f(x)|,\] this is a Banach space. A sequence \(f_n\) in \(C([0,1])\) converges to \(f\) in this norm precisely when \[\|f_n-f\|_\infty\to 0,\] which is exactly uniform convergence on \([0,1]\).

The completeness of \(C([0,1])\) follows from the completeness of the real or complex numbers. If \((f_n)\) is Cauchy in the supremum norm, then for each \(x\in[0,1]\) the sequence \((f_n(x))\) is Cauchy, since \[|f_n(x)-f_m(x)|\leq \|f_n-f_m\|_\infty.\] Thus \(f_n(x)\) converges pointwise to a function \(f(x)\). Moreover, the convergence is uniform: for every \(\varepsilon>0\), there is an \(N\) such that \(\|f_n-f_m\|_\infty<\varepsilon\) whenever \(n,m\geq N\); letting \(m\to\infty\) gives \[|f_n(x)-f(x)|\leq\varepsilon\] for all \(x\in[0,1]\) and all \(n\geq N\). Since a uniform limit of continuous functions is continuous, \(f\in C([0,1])\).

More generally, if \(X\) is a compact Hausdorff space, then \(C(X)\), with the supremum norm, is a Banach space. If \(X\) is not compact, the supremum norm need not be finite on all of \(C(X)\). In that case one often uses the topology of uniform convergence on compact subsets, also called the compact-open topology in this setting. For example, if \(X\) is a locally compact, σ-compact Hausdorff space, this topology is generated by the seminorms

\[p_K(f)=\sup_{x\in K}|f(x)|,\]

where \(K\) ranges over compact subsets of \(X\), making \(C(X)\) into a Fréchet space.

For any set \(X\), the space \(\ell^\infty(X)\) of all bounded real-valued or complex-valued functions on \(X\), equipped with the supremum norm \[\|f\|_\infty=\sup_{x\in X}|f(x)|,\] is also a Banach space. Convergence in this norm is exactly uniform convergence on \(X\). The sup norm on \(C(X)\) for a compact space \(X\) is inherited from the norm of \(\ell^\infty\), so \(C(X)\subset\ell^\infty(X)\) is a closed subspace.

If \((X,\mu)\) is a measure space, the space \(L^\infty(X,\mu)\) of essentially bounded measurable functions on \(X\), together with the essential supremum norm \(\|f\|=\operatorname{ess sup}_{x\in X}|f(x)|\). For many spaces of practical interest, such as compact subsets of Euclidean space and compact topological, the natural Borel measure embeds \(C(X)\) into a closed subspace of \(L^\infty(X,\mu)\). In these cases the norm giving uniform convergence is the same as the \(L^\infty\) norm, and therefore called the \(L^\infty\) norm.