Induced matrix norms
A vector norm measures vector size. The corresponding induced matrix norm measures the largest factor by which a matrix can stretch a vector:
Nakafa
Free learning materials.For AI agents: use /llms.txt for the Nakafa content index.
This definition immediately gives two essential bounds for dimensionally compatible matrices and vectors:
The chosen vector norm determines the induced matrix norm.
| Vector norm | Induced matrix norm |
|---|---|
| , the largest absolute column sum | |
For a square matrix, the spectral radius is
Every induced matrix norm bounds the spectral radius:
The spectral norm satisfies
If is real symmetric, its singular values are the absolute values of its eigenvalues, so
For a general nonnormal matrix, the spectral radius and spectral norm need not be equal.
For an invertible square matrix and an induced norm, define
The condition number is at least one. A singular matrix has infinite condition number. In the Euclidean norm,
For an invertible symmetric matrix, this becomes
Use the eigenvalue with the smallest magnitude in the denominator. Its sign does not decide the choice.
| Size of | Interpretation |
|---|---|
| Near | In the worst case, the relative change in the solution is about as large as the relative change in the input. |
| Large | Some input directions can produce much larger relative changes in the solution. |
| Infinite | The matrix is singular, so the inverse problem is not uniquely solvable. |
Consider
Its singular values are and , hence
For , the solution is
A small change in is amplified by in the second solution component. The matrix is close to singular because it multiplies vectors along the second coordinate direction by , which is close to zero.
Set and compare two right-hand sides. Only the second observation changes.
Assume is invertible and , so the relative errors below have nonzero denominators. Compare the original system with a system whose matrix and vector change slightly:
Use the same induced norm for every quantity. The following bound requires
Under this condition, the relative solution change is bounded by
This bound describes the worst case. Most perturbations may produce less amplification.
Problem conditioning and algorithmic stability describe different properties:
A stable algorithm cannot remove sensitivity already present in an ill-conditioned problem. Conversely, a well-conditioned problem can still be solved poorly by an unstable implementation. A small residual alone also does not guarantee a small forward error when is large.
In numerical work, estimate the reciprocal condition number from a trusted factorization. This gives without explicitly forming .
Published: . Updated: .
| , the largest absolute row sum |
| , the largest singular value |