Spectral Theorem for Complex Matrices

Orthogonal Diagonalization Characterization

The spectral theorem for complex matrices provides a complete characterization of when a complex matrix can be diagonalized using an orthonormal basis of eigenvectors. This is a very important result in linear algebra because it connects geometric concepts (orthonormal basis) with algebraic concepts (matrix normality).

For a complex matrix $A \in \mathbb{C}^{n \times n}$ , the following conditions are equivalent to each other:

There exists an orthonormal basis of $\mathbb{C}^n$ consisting of eigenvectors of matrix $A$
Matrix $A$ is normal

This equivalence is very profound because it shows that algebraic properties (normality) are directly related to the possibility of orthogonal diagonalization.

Proof of the First Direction

Let us prove that if there exists an orthonormal basis of eigenvectors, then the matrix is normal. This is like proving that if a building has a very regular and symmetric structure, then the building has special balance properties.

Let $v_1, \ldots, v_n$ be an orthonormal basis consisting of eigenvectors of matrix $A$ . For each $j = 1, \ldots, n$ we have $Av_j = \lambda_j v_j$ .

Since it's an orthonormal basis, we have the following sequence of calculations:

(A^H v_i)^H v_j = v_i^H (Av_j) = v_i^H (\lambda_j v_j)

= \lambda_j v_i^H v_j = \lambda_j \delta_{ij}

This means $A^H v_i = \overline{\lambda_i} v_i$ , so that:

AA^H v_i = A(\overline{\lambda_i} v_i) = \overline{\lambda_i} \lambda_i v_i

= \lambda_i \overline{\lambda_i} v_i = \lambda_i (A^H v_i) = A^H (\lambda_i v_i) = A^H A v_i

Since this holds for all basis vectors, then $AA^H = A^H A$ , which means $A$ is normal.

Proof from Normality to Diagonalization

Now we prove the opposite direction using mathematical induction. Imagine building a multi-story house, we start from the ground floor and prove that each new floor can be built using results from the previous floor.

Mathematical Induction Structure

Base Step For $n = 0$ (empty matrix), the statement is clearly true.
Induction Hypothesis Assume the statement is true for all normal matrices of size $(n-1) \times (n-1)$ .
Induction Step Let $A \in \mathbb{C}^{n \times n}$ be normal. Based on the fundamental theorem of algebra, there exists an eigenvalue $\lambda_1 \in \mathbb{C}$ of $A$ . Let $v_1 \in \mathbb{C}^n$ be a corresponding eigenvector with $v_1^H v_1 = 1$ .

We have $Av_1 = \lambda_1 v_1$ and from the properties of normal matrices, $A^H v_1 = \overline{\lambda_1} v_1$ .

Formation of Invariant Subspace

Complete $v_1$ to an orthonormal basis of $\mathbb{C}^n$ with vectors $v_2, \ldots, v_n$ . Define:

W = \text{span}(v_2, \ldots, v_n)

Here the span of vectors $v_2, \ldots, v_n$ is the set of all linear combinations of those vectors. In other words, $W$ contains all vectors that can be written as $a_2 v_2 + a_3 v_3 + \cdots + a_n v_n$ with $a_2, a_3, \ldots, a_n \in \mathbb{C}$ .

For every $w \in W$ , we have:

(Aw)^H v_1 = w^H (A^H v_1) = w^H (\overline{\lambda_1} v_1)

= \overline{\lambda_1} w^H v_1 = 0

So $Aw \in W$ , which means:

A(W) = \{Aw \mid w \in W\} \subset W

In other words, $W$ is a subspace that is invariant (unchanging) under transformation $A$ . This is like a separate pond where fish swimming in that pond never leave the pond.

Unitary Transformation and Preservation of Normality

Let $T \in \mathbb{C}^{n \times n}$ be a unitary matrix with columns $v_1, \ldots, v_n$ . Then $T \cdot T^H = T \cdot T^{-1} = I$ .

This transformation gives an elegant block form:

T^H \cdot A \cdot T = T^{-1} \cdot A \cdot T

= \begin{pmatrix} \lambda_1 & 0^T \\ 0 & A' \end{pmatrix}

with:

A' \in \mathbb{C}^{(n-1) \times (n-1)}

Since unitary transformations preserve normality like a mirror that preserves the shape of objects, we can show that:

(T^H \cdot A \cdot T) \cdot (T^H \cdot A \cdot T)^H

= T^H \cdot A \cdot T \cdot T^H \cdot A^H \cdot T

= T^H \cdot A \cdot A^H \cdot T

= T^H \cdot A^H \cdot A \cdot T

= (T^H \cdot A \cdot T)^H \cdot (T^H \cdot A \cdot T)

Therefore the block diagonal matrix is also normal, which means $A'$ is also normal.

Construction of Complete Orthonormal Basis

Now we arrive at the final stage like assembling a puzzle that is almost complete. Based on the induction hypothesis, there exists an orthonormal basis $v'_2, \ldots, v'_n$ of eigenvectors for $A'$ . Let $S'$ be a unitary matrix with those columns, where:

S' \in \mathbb{C}^{(n-1) \times (n-1)}

so that:

S'^H \cdot A' \cdot S' = \begin{pmatrix} \lambda_2 & & \\ & \ddots & \\ & & \lambda_n \end{pmatrix}

with eigenvalues $\lambda_2, \ldots, \lambda_n$ of $A'$ .

Now we define:

S = T \cdot \begin{pmatrix} 1 & 0^T \\ 0 & S' \end{pmatrix}

Then $S$ is a unitary matrix with:

S \in \mathbb{C}^{n \times n}

and:

S^H \cdot A \cdot S = \begin{pmatrix} 1 & 0^T \\ 0 & S'^H \end{pmatrix} \cdot T^H \cdot A \cdot T

\cdot \begin{pmatrix} 1 & 0^T \\ 0 & S' \end{pmatrix}

= \begin{pmatrix} \lambda_1 & 0^T \\ 0 & S'^H A' S' \end{pmatrix}

= \begin{pmatrix} \lambda_1 & & \\ & \ddots & \\ & & \lambda_n \end{pmatrix}

The columns of $S$ form an orthonormal basis of eigenvectors of matrix $A$ .

Real Matrix Case

For real matrices, the situation is slightly different like the difference between drawing on a flat canvas (real) compared to drawing in 3D space (complex). A real matrix:

A \in \mathbb{R}^{n \times n}

is called normal if:

A \cdot A^T = A^T \cdot A

Normal real matrices are a special case of normal complex matrices with real entries. Therefore, the same properties apply to both types of symmetric and orthogonal matrices.

Symmetric matrices and orthogonal matrices are always normal. For a symmetric matrix $A^T = A$ , it is clear that:

A \cdot A^T = A \cdot A = A^T \cdot A

For an orthogonal matrix $A^T = A^{-1}$ , we have:

A \cdot A^T = A \cdot A^{-1} = I

= A^{-1} \cdot A = A^T \cdot A

However, not all normal real matrices have real eigenvalues. In the real spectral theorem, the existence of real eigenvalues is not guaranteed, so orthogonal diagonalization may not always be possible in real numbers.

Command Palette