Best Approximation in Function and Polynomial Spaces

When a Simpler Function Is Enough

A function may take $too long to evaluate$ or be too complicated for a later calculation. We can choose a simpler class of functions, then find the one that is closest to the original function.

Let $C([a,b])$ denote the real vector space of continuous functions on $[a,b]$ . We measure the error across the whole interval.

Assume $a<b$ so the interval has positive length.

Inner Product and Error over an Interval

For $f,g\in C([a,b])$ , define

\langle f,g\rangle=\int_a^b f(x)g(x)\,dx

This inner product turns $C([a,b])$ into an inner-product space. The displayed norm is the $L^2$ norm. Dividing it by the square root of the interval length gives the $root-mean-square value$ :

\operatorname{RMS}(f)=\frac{1}{\sqrt{b-a}}\|f\|_2

The quantity $\|f-g\|_2$ combines the error across the interval. Positive and negative errors cannot cancel because the difference is squared inside the integral.

Best Approximation as an Orthogonal Projection

Let $S\subset C([a,b])$ be a finite-dimensional subspace. The best approximation of $f$ in $S$ is the unique function $g\in S$ satisfying

\|f-g\|_2=\min_{\varphi\in S}\|f-\varphi\|_2

Geometrically, $g$ is the $orthogonal projection$ of $f$ onto $S$ . The residual $r=f-g$ must be orthogonal to every function in the approximation space:

\begin{aligned} &\langle f-g,s\rangle=0 \\ &\text{for every }s\in S \end{aligned}

If $\{\phi_0,\ldots,\phi_n\}$ is a basis of $S$ and $g=\sum_{j=0}^n c_j\phi_j$ , these orthogonality conditions become the normal equations

\begin{aligned} &\sum_{j=0}^n\langle\phi_j,\phi_i\rangle c_j=\langle f,\phi_i\rangle, \\ &i=0,\ldots,n \end{aligned}

Approximating a Quadratic by a Line

Approximate $f(x)=x^2$ on $[-1,1]$ by a function in $S=\operatorname{span}\{1,x\}$ . Write $g(x)=c_0+c_1x$ . Orthogonality to the two basis functions gives

\int_{-1}^{1}\left(x^2-c_0-c_1x\right)\,dx=0

\frac23-2c_0=0,\qquad -\frac23c_1=0\quad\Longrightarrow\quad c_0=\frac13,\ c_1=0

The $odd terms$ integrate to zero on the symmetric interval. The second equation gives $c_1=0$ , and the first gives $c_0=1/3$ . Therefore

g(x)=\frac13

The best line is horizontal. That conclusion is not a guess: the symmetry of $x^2$ forces the coefficient of the odd basis function $x$ to vanish.

The quadratic and its best linear approximation

The cyan parabola is

x^2

, the orange line is

\frac{1}{3}

. Purple segments show signed vertical differences whose squares are integrated.

$x$: $[-1, 1]$
$y$: $[0, 1]$

Polynomial Approximation Spaces

For algebraic polynomials of degree at most $n$ , use

P_n([a,b])=\operatorname{span}\{1,x,x^2,\ldots,x^n\}

Increasing $n$ enlarges the approximation space, so the smallest possible $L^2$ error $cannot increase$ . In computation, orthogonal polynomial bases are often preferable to raw powers because they produce better-conditioned systems.

Trigonometric Approximation Spaces

For period $\omega>0$ , the space $T_n(\omega)$ consists of all functions of the following form, with real coefficients $a_i,b_i$ :

p(x)=\frac{a_0}{2}+\sum_{i=1}^n q_i(x)

Over one full period, these sine and cosine basis functions are $mutually orthogonal$ . Their coefficients can therefore be computed by projecting the target function onto each basis function, which is the finite-dimensional idea behind Fourier approximation.

Choice of $S$	Best suited for	What determines the coefficients
$P_n([a,b])$	Smooth behavior on a finite interval	Orthogonality of the residual to polynomial basis functions
$T_n(\omega)$	Periodic behavior	Orthogonality of the residual to sine and cosine basis functions

A $best approximation is the orthogonal projection$ defined by a chosen function space, interval, and norm. Visual similarity alone does not establish it.