For AI agents: use /llms.txt for the Nakafa content index.
An important question that arises is whether the best approximation really exists and whether its solution is unique? The answer is yes. Let be a Euclidean vector space and be a finite-dimensional vector subspace. Then for every there exists a unique best approximation with
This theorem guarantees that the best approximation always exists and is unique. Like finding the closest point from a location to a highway, there is always one point that gives the shortest distance.
Let be the dimension of and be a basis of . Using the Gram-Schmidt process, we can compute an orthonormal basis of with .
Every has a unique representation as . Then it follows that
Using the identity , we obtain
Function is the best approximation of if and only if for .
For an orthonormal basis of , the best approximation is given by
The best approximation satisfies the distance formula
The best approximation of in is the orthogonal projection of onto . This means
Geometrically, the vector from to is perpendicular to the subspace . Imagine dropping a ball from the air to the floor, the point where it lands is the orthogonal projection of the ball onto the floor.
When an orthonormal basis of is not known, we can use an arbitrary basis of . Let be the unique representation of with respect to this basis.
Since , the orthogonality condition gives
This yields the linear system
The coefficient matrix is called the Gram matrix of the basis . This matrix is symmetric and positive definite. For it holds
However, matrix can become very ill-conditioned in practice. For example, for the monomial basis , the matrix becomes very unstable so that computing becomes difficult for large .
The Gauss approximation with an orthonormal basis of has the advantage of easy computation of the best approximation
without needing to solve a linear system. With an orthonormal basis, we can directly compute the projection coefficients like using a coordinate system that is already neatly arranged and mutually perpendicular.
