Spectral Theorem for Complex Matrices

Unitary Diagonalization Characterization

The complex spectral theorem characterizes exactly when eigenvectors can be chosen orthonormally. It connects the geometry of a unitary basis with the algebraic condition of normality.

For $A\in\mathbb{C}^{n\times n}$ , the following statements are equivalent:

There is an orthonormal basis of $\mathbb{C}^n$ consisting of eigenvectors of $A$ .
The matrix is normal: $A^HA=AA^H$ .
There is a unitary matrix $U$ and a diagonal matrix $\Lambda$ such that $A=U\Lambda U^H$ .

The theorem is stronger than ordinary diagonalizability. A diagonalizable matrix can have a badly conditioned eigenvector basis, whereas $a unitary basis preserves norms and angles$ .

Proof of the First Direction

First, suppose an orthonormal basis of eigenvectors exists. We will show directly that the matrix is normal.

Let $v_1, \ldots, v_n$ be an orthonormal basis consisting of eigenvectors of matrix $A$ . For each $j = 1, \ldots, n$ we have $Av_j = \lambda_j v_j$ .

Since it's an orthonormal basis, we have the following sequence of calculations:

(A^H v_i)^H v_j = v_i^H (Av_j) = v_i^H (\lambda_j v_j)

This means $A^H v_i = \overline{\lambda_i} v_i$ , so that:

AA^H v_i = A(\overline{\lambda_i} v_i) = \overline{\lambda_i} \lambda_i v_i

Since this holds for $all basis vectors$ , then $AA^H = A^H A$ , which means $A$ is normal.

Proof from Normality to Diagonalization

Now we prove the converse by $mathematical induction$ . The base case starts the argument, and the induction step reduces an $n \times n$ matrix to a normal $(n-1) \times (n-1)$ block.

Mathematical Induction Structure

The proof is built by induction, so the steps below fix the base case, state the induction assumption, and then take the step from $n-1$ to $n$ .

$Base Step$ For $n = 0$ , the empty matrix satisfies the statement.

$Induction Hypothesis$ Assume the statement is true for all normal matrices of size $(n-1) \times (n-1)$ .

$Induction Step$ Let $A \in \mathbb{C}^{n \times n}$ be normal. Based on the fundamental theorem of algebra, there exists an eigenvalue $\lambda_1 \in \mathbb{C}$ of $A$ . Let $v_1 \in \mathbb{C}^n$ be a corresponding eigenvector with $v_1^H v_1 = 1$ .

We have $Av_1 = \lambda_1 v_1$ . The following lemma shows that $A^H v_1 = \overline{\lambda_1} v_1$ as well.

Why the Adjoint Has the Same Eigenvector

Let $B=A-\lambda_1I$ . Since $A$ is normal, $B$ is normal, and therefore

\|B^Hv_1\|_2^2=v_1^HBB^Hv_1

Hence $B^Hv_1=0$ , which is exactly $A^Hv_1=\overline{\lambda_1}v_1$ . This identity makes the $orthogonal complement$ of $v_1$ invariant.

Formation of Invariant Subspace

Complete $v_1$ to an orthonormal basis of $\mathbb{C}^n$ with vectors $v_2, \ldots, v_n$ . Define:

W = \operatorname{span}(v_2, \ldots, v_n)

Here the span of vectors $v_2, \ldots, v_n$ is the set of all linear combinations of those vectors. In other words, $W$ contains all vectors that can be written as $a_2 v_2 + a_3 v_3 + \cdots + a_n v_n$ with $a_2, a_3, \ldots, a_n \in \mathbb{C}$ .

For every $w \in W$ , we have:

(Aw)^H v_1 = w^H (A^H v_1) = w^H (\overline{\lambda_1} v_1)

So $Aw \in W$ , which means:

A(W) = \{Aw \mid w \in W\} \subset W

In other words, $W$ is $invariant under$ $A$ : applying $A$ to any vector in $W$ produces another vector in $W$ .

Unitary Transformation and Preservation of Normality

Let $T \in \mathbb{C}^{n \times n}$ be a unitary matrix with columns $v_1, \ldots, v_n$ . Then $T \cdot T^H = T \cdot T^{-1} = I$ .

In this unitary basis, $A$ has the following block form:

T^H \cdot A \cdot T = T^{-1} \cdot A \cdot T

The lower-right block $A'$ has size

A' \in \mathbb{C}^{(n-1) \times (n-1)}

Since unitary similarity transformations $preserve normality$ , we can show that:

(T^H \cdot A \cdot T) \cdot (T^H \cdot A \cdot T)^H

The block diagonal matrix is also normal, which means $A'$ is also normal.

Construction of Complete Orthonormal Basis

By the $induction hypothesis$ , there exists an orthonormal basis $v'_2, \ldots, v'_n$ of eigenvectors for $A'$ . Let $S'$ be a unitary matrix with those columns, where:

S' \in \mathbb{C}^{(n-1) \times (n-1)}

In the basis $S'$ , the matrix $A'$ becomes diagonal:

S'^H \cdot A' \cdot S' = \begin{pmatrix} \lambda_2 & & \\ & \ddots & \\ & & \lambda_n \end{pmatrix}

The diagonal entries $\lambda_2, \ldots, \lambda_n$ are the eigenvalues of $A'$ .

Now we define:

S = T \cdot \begin{pmatrix} 1 & 0^T \\ 0 & S' \end{pmatrix}

Then $S$ is a unitary matrix with:

S \in \mathbb{C}^{n \times n}

Substituting the matrix $S$ into the similarity transformation gives

\begin{aligned} &S^H \cdot A \cdot S \\ &= \begin{pmatrix} 1 & 0^T \\ 0 & S'^H \end{pmatrix} \cdot T^H \cdot A \cdot T \\ &\quad{}\cdot \begin{pmatrix} 1 & 0^T \\ 0 & S' \end{pmatrix} \\ &= \begin{pmatrix} \lambda_1 & 0^T \\ 0 & S'^H A' S' \end{pmatrix} \\ &= \begin{pmatrix} \lambda_1 & & \\ & \ddots & \\ & & \lambda_n \end{pmatrix} \end{aligned}

The columns of $S$ form an orthonormal basis of eigenvectors of matrix $A$ .

Real Matrix Case

For real matrices, the distinction is whether the required eigenvalues and eigenvectors remain real. A real matrix:

A \in \mathbb{R}^{n \times n}

The real matrix $A$ is called normal if

A \cdot A^T = A^T \cdot A

Real normal matrices are still unitarily diagonalizable over $\mathbb{C}$ , but their eigenvectors and eigenvalues need not be real. Over $\mathbb{R}$ , an orthogonal change of basis gives a block diagonal matrix with real eigenvalue blocks $(\lambda)$ and blocks

\begin{pmatrix}a&-b\\b&a\end{pmatrix}

Each $2\times2$ block represents a conjugate eigenvalue pair $a\pm bi$ . A real normal matrix is orthogonally diagonalizable over $\mathbb{R}$ exactly when $all its eigenvalues are real$ .

Symmetric matrices and orthogonal matrices are always normal. For a symmetric matrix $A^T = A$ :

A \cdot A^T = A \cdot A = A^T \cdot A

For an orthogonal matrix $A^T = A^{-1}$ , we have:

A \cdot A^T = A \cdot A^{-1} = I

Rotation Example

A $quarter turn$ is represented by the matrix:

R=\begin{pmatrix}0&-1\\1&0\end{pmatrix}

The matrix $R$ is real and orthogonal, hence normal. Its eigenvalues are $i$ and $-i$ , with the following normalized eigenvectors:

\begin{aligned} &u_+=\frac1{\sqrt2}\begin{pmatrix}1\\-i\end{pmatrix}, \\ &u_-=\frac1{\sqrt2}\begin{pmatrix}1\\i\end{pmatrix} \end{aligned}

$R$ is unitarily diagonalizable over $\mathbb{C}$ but cannot be diagonalized by a real orthogonal matrix. Over $\mathbb{R}$ , the rotation itself is the required $2\times2$ block.