Spectral Theorem

Basic Concepts of Normal Matrices

The spectral theorem answers the important question of when a matrix can be diagonalized using an orthonormal basis of eigenvectors. Imagine you want to transform a complex matrix into a simple diagonal matrix, but using basis vectors that are mutually perpendicular. The spectral theorem provides the precise conditions when this transformation is possible.

When this condition is satisfied, the basis transformation matrix becomes unitary with the property $S^{-1} = S^H$ or orthogonal with the property $S^{-1} = S^T$ for the real case. We will start by studying the complex case first.

A complex matrix $A \in \mathbb{C}^{n \times n}$ is called normal if it satisfies the commutativity condition with its conjugate transpose:

This condition appears simple, but it is actually very powerful. Matrices that can "exchange places" with their conjugate transpose have special geometric properties.

Special Properties of Eigenspaces of Normal Matrices

Normal matrices have interesting properties that arbitrary matrices do not possess. For normal matrices, the null space of the matrix and the null space of its conjugate transpose turn out to be identical.

Let's see why this happens. The null space (kernel) is the set of all vectors $x$ that produce $Ax = 0$. If $A^H A = A A^H$ and $Ax = 0$, then we can analyze it like this:

From this calculation, we conclude that $A^H x = 0$. Therefore, for normal matrices we have $\text{Kern}A^H = \text{Kern}A$.

This equality of null spaces brings important consequences for eigenspaces. For every eigenvalue $\lambda \in \mathbb{C}$, the eigenspaces of $A$ and $A^H$ turn out to be identical.

So every eigenvector of the normal matrix $A$ for eigenvalue $\lambda$ is also an eigenvector of $A^H$ with exactly the same eigenvalue. Imagine finding two mirrors that reflect light in exactly the same direction.

Hermitian and Unitary Matrices as Examples of Normal Matrices

Two important types of matrices that are always normal are Hermitian matrices and unitary matrices. Let's understand why both are special.

Hermitian Matrices and Real Eigenvalues

Hermitian matrices have the property $A^H = A$. Because of the normal definition $A \cdot A^H = A^H \cdot A$, for Hermitian matrices we have $A \cdot A = A \cdot A$, which is clearly always true.

Eigenvalues of Hermitian matrices are always real. To understand this, we use the fact that for normal matrices, the eigenspaces of $A$ and $A^H$ for the same eigenvalue are identical.

The condition $\lambda = \overline{\lambda}$ means the eigenvalue equals its complex conjugate, which only happens if $\lambda$ is a pure real number. So all eigenvalues of Hermitian matrices are always real numbers, not complex numbers with imaginary parts.

Unitary Matrices and Eigenvalues on the Unit Circle

Unitary matrices have the property $A^H = A^{-1}$. To prove that unitary matrices are also normal, we substitute into the definition and get $A \cdot A^H = A \cdot A^{-1} = I = A^{-1} \cdot A = A^H \cdot A$.

Eigenvalues of unitary matrices have magnitude 1, meaning they lie on the unit circle in the complex plane. We can show this with the following calculation.

The condition $\lambda \cdot \overline{\lambda} = 1$ is mathematically equivalent to $|\lambda|^2 = 1$, which means $|\lambda| = 1$. So all eigenvalues of unitary matrices have modulus exactly equal to 1. This modulus is the distance from the origin in the complex plane. Imagine a spinning wheel, unitary transformations only rotate vectors without changing their length.