# Nakafa Learning Content

> For AI agents: use [llms.txt](https://nakafa.com/llms.txt) for the site index. Markdown versions are available by appending `.md` to content URLs or sending `Accept: text/markdown`.

URL: https://nakafa.com/en/subjects/ai-ds/linear-methods/spectral-theorem
Source: https://raw.githubusercontent.com/nakafaai/nakafa.com/refs/heads/main/packages/contents/material/lesson/ai-ds/linear-methods/spectral-theorem/en.mdx

Learn when matrices can be diagonalized with orthonormal eigenvectors. Learn normal, Hermitian, and unitary matrices with real-world applications.

---

## Basic Concepts of Normal Matrices

The spectral theorem answers the important question of when a matrix can be diagonalized using an orthonormal basis of eigenvectors. Imagine you want to transform a complex matrix into a simple diagonal matrix, but using basis vectors that are mutually perpendicular. The spectral theorem provides the precise conditions when this transformation is possible.

When this condition is satisfied, the basis transformation matrix becomes unitary with the property $$S^{-1} = S^H$$ or orthogonal with the property $$S^{-1} = S^T$$ for the real case. We will start by studying the complex case first.

Visible text: When this condition is satisfied, the basis transformation matrix becomes unitary with the property or orthogonal with the property for the real case. We will start by studying the complex case first.

A complex matrix $$A \in \mathbb{C}^{n \times n}$$ is called normal if it satisfies the commutativity condition with its conjugate transpose:

Visible text: A complex matrix is called normal if it satisfies the commutativity condition with its conjugate transpose:

```math
A \cdot A^H = A^H \cdot A
```

This condition appears simple, but it has a strong geometric meaning. Matrices that can "exchange places" with their conjugate transpose have special geometric properties.

## Special Properties of Eigenspaces of Normal Matrices

Normal matrices have interesting properties that arbitrary matrices do not possess. For normal matrices, the null space of the matrix and the null space of its conjugate transpose turn out to be identical.

Let's see why this happens. The null space (kernel) is the set of all vectors $$x$$ that produce $$Ax = 0$$. If $$A^H A = A A^H$$ and $$Ax = 0$$, then we can analyze it like this:

Visible text: Let's see why this happens. The null space (kernel) is the set of all vectors that produce . If and , then we can analyze it like this:

Component: MathContainer
Children:

```math
0 = (Ax)^H (Ax) = x^H A^H Ax = x^H A A^H x = (A^H x)^H (A^H x)
```

From this calculation, we conclude that $$A^H x = 0$$. Therefore, for normal matrices we have $$\text{Kern}A^H = \text{Kern}A$$.

Visible text: From this calculation, we conclude that . Therefore, for normal matrices we have .

This equality of null spaces brings important consequences for eigenspaces. For every eigenvalue $$\lambda \in \mathbb{C}$$, the eigenspaces of $$A$$ and $$A^H$$ turn out to be identical.

Visible text: This equality of null spaces brings important consequences for eigenspaces. For every eigenvalue , the eigenspaces of and turn out to be identical.

```math
\text{Eig}_A(\lambda) = \text{Eig}_{A^H}(\lambda)
```

So every eigenvector of the normal matrix $$A$$ for eigenvalue $$\lambda$$ is also an eigenvector of $$A^H$$ with exactly the same eigenvalue. Imagine finding two mirrors that reflect light in exactly the same direction.

Visible text: So every eigenvector of the normal matrix for eigenvalue is also an eigenvector of with exactly the same eigenvalue. Imagine finding two mirrors that reflect light in exactly the same direction.

## Hermitian and Unitary Matrices as Examples of Normal Matrices

Two important types of matrices that are always normal are Hermitian matrices and unitary matrices. Let's understand why both are special.

### Hermitian Matrices and Real Eigenvalues

Hermitian matrices have the property $$A^H = A$$. Because of the normal definition $$A \cdot A^H = A^H \cdot A$$, for Hermitian matrices we have $$A \cdot A = A \cdot A$$, which is clearly always true.

Visible text: Hermitian matrices have the property . Because of the normal definition , for Hermitian matrices we have , which is clearly always true.

Eigenvalues of Hermitian matrices are always real. To understand this, we use the fact that for normal matrices, the eigenspaces of $$A$$ and $$A^H$$ for the same eigenvalue are identical.

Visible text: Eigenvalues of Hermitian matrices are always real. To understand this, we use the fact that for normal matrices, the eigenspaces of and for the same eigenvalue are identical.

```math
\text{Eig}_A(\lambda) = \text{Eig}_{A^H}(\lambda) = \text{Eig}_A(\overline{\lambda}) \Rightarrow \lambda = \overline{\lambda}
```

The condition $$\lambda = \overline{\lambda}$$ means the eigenvalue equals its complex conjugate, which only happens if $$\lambda$$ is a pure real number. So all eigenvalues of Hermitian matrices are always real numbers, not complex numbers with imaginary parts.

Visible text: The condition means the eigenvalue equals its complex conjugate, which only happens if is a pure real number. So all eigenvalues of Hermitian matrices are always real numbers, not complex numbers with imaginary parts.

### Unitary Matrices and Eigenvalues on the Unit Circle

Unitary matrices have the property $$A^H = A^{-1}$$. To prove that unitary matrices are also normal, we substitute into the definition and get $$A \cdot A^H = A \cdot A^{-1} = I = A^{-1} \cdot A = A^H \cdot A$$.

Visible text: Unitary matrices have the property . To prove that unitary matrices are also normal, we substitute into the definition and get .

Eigenvalues of unitary matrices have magnitude $$1$$, meaning they lie on the unit circle in the complex plane. We can show this with the following calculation.

Visible text: Eigenvalues of unitary matrices have magnitude , meaning they lie on the unit circle in the complex plane. We can show this with the following calculation.

Component: MathContainer
Children:

```math
\text{Eig}_A(\lambda) = \text{Eig}_{A^H}(\lambda) = \text{Eig}_{A^{-1}}(\lambda) = \text{Eig}_A(\lambda^{-1})
```

```math
\Rightarrow \lambda = \lambda^{-1} \Rightarrow \lambda \cdot \overline{\lambda} = 1
```

The condition $$\lambda \cdot \overline{\lambda} = 1$$ is mathematically equivalent to $$|\lambda|^2 = 1$$, which means $$|\lambda| = 1$$. So all eigenvalues of unitary matrices have modulus exactly equal to $$1$$. This modulus is the distance from the origin in the complex plane. Imagine a spinning wheel, unitary transformations only rotate vectors without changing their length.

Visible text: The condition is mathematically equivalent to , which means . So all eigenvalues of unitary matrices have modulus exactly equal to . This modulus is the distance from the origin in the complex plane. Imagine a spinning wheel, unitary transformations only rotate vectors without changing their length.