The spectral theorem for complex matrices provides a complete characterization of when a complex matrix can be diagonalized using an orthonormal basis of eigenvectors. This is a very important result in linear algebra because it connects geometric concepts (orthonormal basis) with algebraic concepts (matrix normality).
For a complex matrix A∈Cn×n, the following conditions are equivalent to each other:
There exists an orthonormal basis of consisting of eigenvectors of matrix
Cn
A
Matrix A is normal
This equivalence is important because it shows that algebraic properties (normality) are directly related to the possibility of orthogonal diagonalization.
Let us prove that if there exists an orthonormal basis of eigenvectors, then the matrix is normal. This is like proving that if a building has a very regular and symmetric structure, then the building has special balance properties.
Let v1,…,vn be an orthonormal basis consisting of eigenvectors of matrix A. For each j=1,…,n we have Avj=λjvj.
Since it's an orthonormal basis, we have the following sequence of calculations:
(AHvi)Hvj=viH(Avj)=viH(λjvj)
=λjviHvj=λjδij
This means AHvi=λivi, so that:
AAHvi=A(λivi)=λiλivi
=λiλivi=λi(AHvi)=AH(λivi)=AHAvi
Since this holds for all basis vectors, then AAH=AHA, which means A is normal.
Now we prove the opposite direction using mathematical induction. Imagine building a multi-story house, we start from the ground floor and prove that each new floor can be built using results from the previous floor.
Base Step
For n=0 (empty matrix), the statement is clearly true.
Induction Hypothesis
Assume the statement is true for all normal matrices of size (n−1)×(n−1).
Induction Step
Let A∈Cn×n be normal. Based on the fundamental theorem of algebra, there exists an eigenvalue λ1∈C of A. Let v1∈Cn be a corresponding eigenvector with v1Hv1=1.
We have Av1=λ1v1 and from the properties of normal matrices, AHv1=λ1v1.
Complete v1 to an orthonormal basis of Cn with vectors v2,…,vn. Define:
W=span(v2,…,vn)
Here the span of vectors v2,…,vn is the set of all linear combinations of those vectors. In other words, W contains all vectors that can be written as a2v2+a3v3+⋯+anvn with a2,a3,…,an∈C.
For every w∈W, we have:
(Aw)Hv1=wH(AHv1)=wH(λ1v1)
=λ1wHv1=0
So Aw∈W, which means:
A(W)={Aw∣w∈W}⊂W
In other words, W is a subspace that is invariant (unchanging) under transformation A. This is like a separate pond where fish swimming in that pond never leave the pond.
Now we arrive at the final stage like assembling a puzzle that is almost complete. Based on the induction hypothesis, there exists an orthonormal basis v2′,…,vn′ of eigenvectors for A′. Let S′ be a unitary matrix with those columns, where:
S′∈C(n−1)×(n−1)
so that:
S′H⋅A′⋅S′=λ2⋱λn
with eigenvalues λ2,…,λn of A′.
Now we define:
S=T⋅(100TS′)
Then S is a unitary matrix with:
S∈Cn×n
and:
SH⋅A⋅S=(100TS′H)⋅TH⋅A⋅T
⋅(100TS′)
=(λ100TS′HA′S′)
=λ1⋱λn
The columns of S form an orthonormal basis of eigenvectors of matrix A.
For real matrices, the situation is slightly different like the difference between drawing on a flat canvas (real) compared to drawing in three-dimensional space (complex). A real matrix:
A∈Rn×n
is called normal if:
A⋅AT=AT⋅A
Normal real matrices are a special case of normal complex matrices with real entries. Therefore, the same properties apply to both types of symmetric and orthogonal matrices.
Symmetric matrices and orthogonal matrices are always normal. For a symmetric matrix AT=A, it is clear that:
A⋅AT=A⋅A=AT⋅A
For an orthogonal matrix AT=A−1, we have:
A⋅AT=A⋅A−1=I
=A−1⋅A=AT⋅A
However, not all normal real matrices have real eigenvalues. In the real spectral theorem, the existence of real eigenvalues is not guaranteed, so orthogonal diagonalization may not always be possible in real numbers.
