Polynomial interpolation

In numerical analysis, polynomial interpolation is the interpolation of a given data set by the polynomial of lowest possible degree that passes through the points in the dataset.

Given a set of n + 1 data points \((x_0,y_0), \ldots, (x_n,y_n)\), with no two \(x_j\) the same, a polynomial function \(p(x)=a_0+a_1x+\cdots+a_nx^n\) is said to interpolate the data if \(p(x_j)=y_j\) for each \(j\in\{0,1,\dotsc,n\}\).

There is always a unique such polynomial, commonly given by two explicit formulas, the Lagrange polynomials and Newton polynomials.

The original use of interpolation polynomials was to approximate values of important transcendental functions such as natural logarithm and trigonometric functions. Starting with a few accurately computed data points, the corresponding interpolation polynomial will approximate the function at an arbitrary nearby point. Polynomial interpolation also forms the basis for algorithms in numerical quadrature (Simpson's rule) and numerical ordinary differential equations (multigrid methods).

In computer graphics, polynomials can be used to approximate complicated plane curves given a few specified points, for example the shapes of letters in typography. This is usually done with Bézier curves, which are a simple generalization of interpolation polynomials (having specified tangents as well as specified points).

In numerical analysis, polynomial interpolation is essential to perform sub-quadratic multiplication and squaring, such as Karatsuba multiplication and Toom-Cook multiplication, where interpolation through points on a product polynomial yields the specific product required. For example, given a = f(x) = a₀x + a₁x + ··· and b = g(x) = b₀x + b₁x + ···, the product ab is a specific value of W(x) = f(x)g(x). One may easily find points along W(x) at small values of x, and interpolation based on those points will yield the terms of W(x) and the specific product ab. As fomulated in Karatsuba multiplication, this technique is substantially faster than quadratic multiplication, even for modest-sized inputs, especially on parallel hardware.

In computer science, polynomial interpolation also leads to algorithms for secure multi party computation and secret sharing.

For any \(n+1\) bivariate data points \((x_0,y_0),\dotsc,(x_n,y_n) \in \R^2\), where no two \(x_j\) are the same, there exists a unique polynomial \(p(x)\) of degree at most \(n\) that interpolates these points, i.e. \(p(x_0)=y_0, \ldots, p(x_n)=y_n\).

Equivalently, for a fixed choice of interpolation nodes \(x_j\), polynomial interpolation defines a linear bijection \(L_n\) between the (n+1)-tuples of real-number values \((y_0,\ldots,y_n)\in \R^{n+1}\) and the vector space \(P(n)\) of real polynomials of degree at most n: \[L_n : \mathbb{R}^{n+1} \stackrel{\sim}{\longrightarrow}\, P(n).\]

This is a type of unisolvence theorem. The theorem is also valid over any infinite field in place of the real numbers \(\R\), for example the rational or complex numbers.

Consider the Lagrange basis functions \(L_0(x),\ldots,L_n(x)\) given by: \[L_j(x)=\prod_{i=0,i\neq j}^{n}\frac{x-x_i}{x_j-x_i} = \frac{(x-x_0)\cdots(x-x_{j-1})(x-x_{j+1})\cdots(x-x_n)} {(x_j-x_0)\cdots(x_j-x_{j-1})(x_j-x_{j+1})\cdots(x_j-x_n)}.\]

Notice that \(L_j(x)\) is a polynomial of degree \(n\), and we have \(L_j(x_k)=0\) for each \(j\neq k\), while \(L_k(x_k)=1\). It follows that the linear combination: \[p(x) = \sum_{j=0}^n y_j L_j(x)\] has \(p(x_k)=\sum_j y_j \,L_j(x_k) = y_k\), so \(p(x)\) is an interpolating polynomial of degree \(n\).

To prove uniqueness, assume that there exists another interpolating polynomial \(q(x)\) of degree at most \(n\), so that \(p(x_k)=q(x_k)\) for all \(k=0,\dotsc,n\). Then \(p(x)-q(x)\) is a polynomial of degree at most \(n\) which has \(n+1\) distinct zeros (the \(x_k\)). But a non-zero polynomial of degree at most \(n\) can have at most \(n\) zeros, so \(p(x)-q(x)\) must be the zero polynomial, i.e. \(p(x)=q(x)\).

Write out the interpolation polynomial in the form

Substituting this into the interpolation equations \(p(x_j) = y_j\), we get a system of linear equations in the coefficients \(a_j\), which reads in matrix-vector form as the following multiplication: \[\begin{bmatrix} x_0^n & x_0^{n-1} & x_0^{n-2} & \ldots & x_0 & 1 \\ x_1^n & x_1^{n-1} & x_1^{n-2} & \ldots & x_1 & 1 \\ \vdots & \vdots & \vdots & & \vdots & \vdots \\ x_n^n & x_n^{n-1} & x_n^{n-2} & \ldots & x_n & 1 \end{bmatrix} \begin{bmatrix} a_n \\ a_{n-1} \\ \vdots \\ a_0 \end{bmatrix} = \begin{bmatrix} y_0 \\ y_1 \\ \vdots \\ y_n \end{bmatrix}.\]

An interpolant \(p(x)\) corresponds to a solution \(A = (a_n,\ldots,a_0)\) of the above matrix equation \(X \cdot A = Y\). The matrix X on the left is a Vandermonde matrix, whose determinant is known to be \(\textstyle \det(X) = \prod_{0 \le i < j \le n} (x_j - x_i),\) which is non-zero since the nodes \(x_j\) are all distinct. This ensures that the matrix is invertible and the equation has the unique solution \(A = X^{-1}\cdot Y\); that is, \(p(x)\) exists and is unique.

We may write down the polynomial immediately in terms of Lagrange polynomials as: \[\begin{align} p(x) &= \frac{(x-x_1)(x-x_2)\cdots(x-x_n)}{(x_0-x_1)(x_0-x_2)\cdots(x_0-x_n)} y_0 \\ [4pt] &+ \frac{(x-x_0)(x-x_2)\cdots(x-x_n)}{(x_1-x_0)(x_1-x_2) \cdots(x_1-x_n)}y_1 \\ [4pt] &+ \cdots\\ [4pt] &+\frac{(x-x_0)(x-x_1)\cdots(x-x_{n-1})}{(x_n-x_0)(x_n-x_1)\cdots(x_n-x_{n-1})}y_n \\ [7pt] &=\sum_{i=0}^n \Biggl( \prod_{\stackrel{\!0\,\leq\, j\,\leq\, n}{j\,\neq\, i}} \frac{x-x_j}{x_i-x_j} \Biggr) y_i =\sum_{i=0}^n \frac{p(x)}{p'(x_i)(x-x_i)}\,y_i \end{align}\]For matrix arguments, this formula is called Sylvester's formula and the matrix-valued Lagrange polynomials are the Frobenius covariants.

A Lozenge diagram is a diagram that is used to describe different interpolation formulas that can be constructed for a given data set. A line starting on the left edge and tracing across the diagram to the right can be used to represent an interpolation formula if the following rules are followed:

1. Left to right steps indicate addition whereas right to left steps indicate subtraction
2. If the slope of a step is positive, the term to be used is the product of the difference and the factor immediately below it. If the slope of a step is negative, the term to be used is the product of the difference and the factor immediately above it.
3. If a step is horizontal and passes through a factor, use the product of the factor and the average of the two terms immediately above and below it. If a step is horizontal and passes through a difference, use the product of the difference and the average of the two terms immediately above and below it.

The factors are expressed using the formula:\[C(u+k,n)=\frac{(u+k)(u+k-1)\cdots(u+k-n+1)}{n!}\]

The Vandermonde matrix in the second proof above may have large condition number, causing large errors when computing the coefficients a_i if the system of equations is solved using Gaussian elimination.

Several authors have therefore proposed algorithms which exploit the structure of the Vandermonde matrix to compute numerically stable solutions in O(n) operations instead of the O(n) required by Gaussian elimination. These methods rely on constructing first a Newton interpolation of the polynomial and then converting it to a monomial form.

To find the interpolation polynomial p(x) in the vector space P(n) of polynomials of degree n, we may use the usual monomial basis for P(n) and invert the Vandermonde matrix by Gaussian elimination, giving a computational cost of O(n) operations. To improve this algorithm, a more convenient basis for P(n) can simplify the calculation of the coefficients, which must then be translated back in terms of the monomial basis.

One method is to write the interpolation polynomial in the Newton form (i.e. using Newton basis) and use the method of divided differences to construct the coefficients, e.g. Neville's algorithm. The cost is O(n) operations. Furthermore, you only need to do O(n) extra work if an extra point is added to the data set, while for the other methods, you have to redo the whole computation.

Another method is preferred when the aim is not to compute the coefficients of p(x), but only a single value p(a) at a point x = a not in the original data set. The Lagrange form computes the value p(a) with complexity O(n).

The Bernstein form was used in a constructive proof of the Weierstrass approximation theorem by Bernstein and has gained great importance in computer graphics in the form of Bézier curves.

Given a set of (position, value) data points \((x_0, y_0), \ldots, (x_j, y_j), \ldots, (x_n, y_n)\) where no two positions \(x_j\) are the same, the interpolating polynomial \(y(x)\) may be considered as a linear combination of the values \(y_j\), using coefficients which are polynomials in \(x\) depending on the \(x_j\). For example, the interpolation polynomial in the Lagrange form is the linear combination \[y(x) := \sum_{j=0}^{k} y_j c_j(x)\] with each coefficient \(c_j(x)\) given by the corresponding Lagrange basis polynomial on the given positions \(x_j\): \[c_j(x) = L_j(x_0,\ldots,x_n;x) = \prod_{ 0 \le i \le n \atop i \neq j } \frac{x-x_i}{x_j-x_i} = \frac{(x-x_0)}{(x_j-x_0)} \cdots \frac{(x-x_{j-1})}{(x_j-x_{j-1})} \frac{(x-x_{j+1})}{(x_j-x_{j+1})} \cdots \frac{(x-x_n)}{(x_j-x_n)}.\]

Since the coefficients depend only on the positions \(x_j\), not the values \(y_j\), we can use the same coefficients to find the interpolating polynomial for a second set of data points \((x_0, v_0), \ldots, (x_n, v_n)\) at the same positions: \[v(x) := \sum_{j=0}^{k} v_j c_j(x).\]

Furthermore, the coefficients \(c_j(x)\) only depend on the relative spaces \(x_i-x_j\) between the positions. Thus, given a third set of data whose points are given by the new variable \(t = ax+b\) (an affine transformation of \(x\), inverted by \(x=\tfrac{t-b}{a}\)): \[(t_0, w_0), \ldots, (t_j, w_j) \ldots, (t_n, w_n) \qquad \text{with}\qquad t_j = ax_j + b,\]

we can use a transformed version of the previous coefficient polynomials:

and write the interpolation polynomial as:

Data points \((x_j,y_j)\) often have equally spaced positions, which may be normalized by an affine transformation to \(x_j = j\). For example, consider the data points

The interpolation polynomial in the Lagrange form is the linear combination

Condensed: the full section is in Wikipedia.

When interpolating a given function f by a polynomial \(p_n\) of degree n at the nodes x₀,..., x_n we get the error \[f(x) - p_n(x) = f[x_0,\ldots,x_n,x] \prod_{i=0}^n (x-x_i)\]

where \(f[x_0,\ldots,x_n,x]\) is the (n+1) divided difference of the data points

Furthermore, there is a Lagrange remainder form of the error, for a function f which is n + 1 times continuously differentiable on a closed interval \(I\), and a polynomial \(p_n(x)\) of degree at most n that interpolates f at n + 1 distinct points \(x_0,\ldots,x_n\in I\). For each \(x\in I\) there exists \(\xi\in I\) such that

\[f(x) - p_n(x) = \frac{f^{(n+1)}(\xi)}{(n+1)!} \prod_{i=0}^n (x-x_i).\]

This error bound suggests choosing the interpolation points x_i to minimize the product \(\left | \prod (x - x_i) \right |\), which is achieved by the Chebyshev nodes.

Set the error term as \(R_n(x) = f(x) - p_n(x)\), and define an auxiliary function:\[Y(t) = R_n(t) - \frac{R_n(x)}{W(x)} W(t) \qquad\text{where}\qquad W(t) = \prod_{i=0}^n (t-x_i).\]Thus:\[Y^{(n+1)}(t) = R_n^{(n+1)}(t) - \frac{R_n(x)}{W(x)} \ (n+1)!\]

But since \(p_n(x)\) is a polynomial of degree at most n, we have \(R_n^{(n+1)}(t) = f^{(n+1)}(t)\), and: \[Y^{(n+1)}(t) = f^{(n+1)}(t) - \frac{R_n(x)}{W(x)} \ (n+1)!\]

Now, since x_i are roots of \(R_n(t)\) and \(W(t)\), we have \(Y(x)=Y(x_j)=0\), which means Y has at least n + 2 roots. From Rolle's theorem, \(Y^\prime(t)\) has at least n + 1 roots, and iteratively \(Y^{(n+1)}(t)\) has at least one root ξ in the interval I. Thus: \[Y^{(n+1)}(\xi) = f^{(n+1)}(\xi) - \frac{R_n(x)}{W(x)} \ (n+1)! = 0\]

and: \[R_n(x) = f(x) - p_n(x) = \frac{f^{(n+1)}(\xi)}{(n+1)!} \prod_{i=0}^n (x-x_i) .\]

This parallels the reasoning behind the Lagrange remainder term in the Taylor theorem; in fact, the Taylor remainder is a special case of interpolation error when all interpolation nodes x_i are identical. Note that the error will be zero when \(x = x_i\) for any i. Thus, the maximum error will occur at some point in the interval between two successive nodes.

In the case of equally spaced interpolation nodes where \(x_i = a + ih\), for \(i=0,1,\ldots,n,\) and where \(h = (b-a)/n,\) the product term in the interpolation error formula can be bound as \[\left|\prod_{i=0}^n (x-x_i)\right| = \prod_{i=0}^n \left|x-x_i\right| \leq \frac{n!}{4} h^{n+1}.\]

Thus the error bound can be given as \[\left|R_n(x)\right| \leq \frac{h^{n+1}}{4(n+1)} \max_{\xi\in[a,b]} \left|f^{(n+1)}(\xi)\right|\]

However, this assumes that \(f^{(n+1)}(\xi)\) is dominated by \(h^{n+1}\), i.e. \(f^{(n+1)}(\xi) h^{n+1} \ll 1\). In several cases, this is not true and the error actually increases as n → ∞ (see Runge's phenomenon). That question is treated in the section Convergence properties.

We fix the interpolation nodes x₀, ..., x_n and an interval [a, b] containing all the interpolation nodes. The process of interpolation maps the function f to a polynomial p. This defines a mapping X from the space C([a, b]) of all continuous functions on [a, b] to itself. The map X is linear and it is a projection on the subspace \(P(n)\) of polynomials of degree n or less.

The Lebesgue constant L is defined as the operator norm of X. One has (a special case of Lebesgue's lemma): \[\left\|f-X(f)\right\| \le (L+1) \left\|f-p^*\right\|.\]

In other words, the interpolation polynomial is at most a factor (L + 1) worse than the best possible approximation. This suggests that we look for a set of interpolation nodes that makes L small. In particular, we have for Chebyshev nodes: \[L \le \frac2\pi \log(n+1) + 1.\]

We conclude again that Chebyshev nodes are a very good choice for polynomial interpolation, as the growth in n is exponential for equidistant nodes. However, those nodes are not optimal.