For AI agents: use /llms.txt for the Nakafa content index.
Imagine we have a bowl that always faces upward. No matter from which direction we throw a ball into it, the ball will always roll to the lowest point. Positive definite matrices have a property similar to this bowl in mathematical space.
A symmetric matrix or Hermitian matrix is called if:
A matrix is called positive definite if a stronger condition is satisfied:
Conversely, a matrix is called negative semidefinite if is positive semidefinite, and negative definite if is positive definite. A matrix that is neither positive nor negative semidefinite is called indefinite.
Why is this concept important? Let's look at it from an interesting geometric perspective.
If is a positive definite matrix, then the set:
forms an ellipsoid in -dimensional space centered at the origin. This ellipsoid shape provides a visual representation of how the matrix "stretches" space in various directions.
Specifically, if where is the identity matrix, then becomes a sphere with radius .
One simple but important property of positive definite matrices is that all their diagonal elements must be positive.
If is a positive definite matrix, then all diagonal elements for .
Why is this so? Because if we take the standard basis vector that has component at position and elsewhere, then:
However, this condition is not sufficient to guarantee positive definiteness. We can have a matrix with all positive diagonal elements but still not be positive definite.
The most elegant way to determine positive definiteness is through eigenvalues. Let's look at this very useful criterion.
A symmetric matrix or Hermitian matrix is positive definite if and only if all its eigenvalues are positive:
For positive semidefinite, all eigenvalues must be non-negative ().
Why is this true? Because for symmetric or Hermitian matrices, we can perform orthogonal diagonalization. If where is a diagonal matrix containing eigenvalues, then:
where . This expression is positive for all if and only if all .
There's another practical way to check positive definiteness without computing eigenvalues. This method is called the leading principal minor criteria or Hauptminorenkriterium.
Let be a symmetric matrix. For , define the -th leading principal minor as:
This is the upper-left submatrix of size from matrix . The determinant of is called the -th leading principal minor.
A symmetric matrix is positive definite if and only if all its leading principal minors are positive:
It's important to note that this leading principal minor criterion only detects positive definiteness, not positive semidefiniteness.
Let's look at some examples to better understand these concepts.
Consider the matrix . This matrix has eigenvalues approximately and .
Since there is a negative eigenvalue, this matrix is indefinite. Even though its diagonal elements ( and ) are both positive, this doesn't guarantee positive definiteness.
Now consider the matrix . This matrix has eigenvalues and . Both eigenvalues are positive, so this matrix is positive definite.
We can also verify this using the leading principal minor criteria:
Since all leading principal minors are positive, this matrix is positive definite.
From the previous example, the inverse of the positive definite matrix is:
This matrix has eigenvalues approximately and . Both are positive, so is also positive definite.
One important result in linear algebra is the property of the matrix for rectangular matrices.
If with , then the matrix is positive semidefinite. This matrix becomes positive definite if and only if has full rank (rank ).
Why is this so? Because for any vector :
This expression equals zero only if . If has full rank, then only for , so is positive definite.
A very useful concept in practice is the ability to "shift" the spectrum of a matrix.
If is a symmetric matrix or is a Hermitian matrix, and is a real number smaller than all eigenvalues of , then the matrix:
is positive definite.
This provides a practical way to make a matrix positive definite by shifting its eigenvalues. If we know the lower bound of the smallest eigenvalue, we can shift the spectrum so that all eigenvalues become positive.
Positive definite matrices play a central role in optimization, numerical analysis, and machine learning due to their geometric properties that guarantee the existence of a unique global minimum.
