# Nakafa Learning Content

> For AI agents: use [llms.txt](https://nakafa.com/llms.txt) for the site index. Markdown versions are available by appending `.md` to content URLs or sending `Accept: text/markdown`.

URL: https://nakafa.com/en/subjects/ai-ds/linear-methods/diagonalization-matrix
Source: https://raw.githubusercontent.com/nakafaai/nakafa.com/refs/heads/main/packages/contents/material/lesson/ai-ds/linear-methods/diagonalization-matrix/en.mdx

Transform matrices using eigenvalues and eigenvectors. Learn diagonalization conditions, geometric vs algebraic multiplicity for AI computations.

---

## Matrix Diagonalization Concept

In matrix theory, we often seek ways to simplify matrix forms to make them easier to analyze and compute. Diagonalization is one technique for doing this. Imagine transforming a complex space into a more orderly space where each dimension does not interfere with each other.

The main goal of diagonalization is to find a special basis so that the linear transformation $$y = A \cdot x$$ can be represented through a diagonal matrix $$B = S^{-1} \cdot A \cdot S$$. If this basis is an orthonormal basis, then the transformation matrix has the property $$S^{-1} = S^T$$.

Visible text: The main goal of diagonalization is to find a special basis so that the linear transformation can be represented through a diagonal matrix . If this basis is an orthonormal basis, then the transformation matrix has the property .

## Definition of Diagonalization

A matrix $$A \in \mathbb{K}^{n \times n}$$ is called **diagonalizable** if it is similar to some diagonal matrix $$\Lambda \in \mathbb{K}^{n \times n}$$, that is, if there exists an invertible matrix $$S \in \mathbb{K}^{n \times n}$$ such that:

Visible text: A matrix is called **diagonalizable** if it is similar to some diagonal matrix , that is, if there exists an invertible matrix such that:

```math
\Lambda = S^{-1} \cdot A \cdot S
```

## Basic Conditions for Diagonalization

When can a matrix $$A \in \mathbb{K}^{n \times n}$$ be diagonalized? The answer is when we can find a basis of $$\mathbb{K}^n$$ that consists entirely of eigenvectors $$v_1, \ldots, v_n \in \mathbb{K}^n$$ of $$A$$ with corresponding eigenvalues $$\lambda_1, \ldots, \lambda_n \in \mathbb{K}$$.

Visible text: When can a matrix be diagonalized? The answer is when we can find a basis of that consists entirely of eigenvectors of with corresponding eigenvalues .

The diagonal matrix $$\Lambda$$ is:

Visible text: The diagonal matrix is:

```math
\Lambda = \begin{pmatrix} \lambda_1 & & \\ & \ddots & \\ & & \lambda_n \end{pmatrix} = \text{diag}(\lambda_1, \ldots, \lambda_n)
```

and $$S$$ is the matrix with columns:

Visible text: and is the matrix with columns:

```math
S = (v_1 \quad \ldots \quad v_n)
```

If $$A$$ is diagonalizable, then the columns $$v_1, \ldots, v_n$$ of $$S$$ form a basis of eigenvectors. From $$\Lambda = S^{-1} \cdot A \cdot S$$ we obtain $$A \cdot S = S \cdot \Lambda$$ and thus $$A \cdot v_i = \lambda_i \cdot v_i$$ for $$i = 1, \ldots, n$$.

Visible text: If is diagonalizable, then the columns of form a basis of eigenvectors. From we obtain and thus for .

Conversely, if $$v_1, \ldots, v_n$$ is a basis of eigenvectors, then $$S$$ is invertible and from $$A \cdot v_i = \lambda_i \cdot v_i$$ for $$i = 1, \ldots, n$$ we obtain $$A \cdot S = S \cdot \Lambda$$ and thus $$\Lambda = S^{-1} \cdot A \cdot S$$.

Visible text: Conversely, if is a basis of eigenvectors, then is invertible and from for we obtain and thus .

## Example of Non-Diagonalizable Case

Consider the matrix:

```math
A = \begin{pmatrix} 1 & 2 \\ 0 & 1 \end{pmatrix}
```

This matrix has eigenvalue $$\lambda = 1$$ with algebraic multiplicity $$\mu_A(1) = 2$$. The eigenspace is the kernel (null space) of $$A - 1 \cdot I$$:

Visible text: This matrix has eigenvalue with algebraic multiplicity . The eigenspace is the kernel (null space) of :

Component: MathContainer
Children:

```math
\text{Eig}_A(1) = \text{Kern}(A - 1 \cdot I)
```

```math
= \text{Kern}\begin{pmatrix} 0 & 2 \\ 0 & 0 \end{pmatrix}
```

```math
= \text{Span}\begin{pmatrix} 1 \\ 0 \end{pmatrix}
```

which has dimension $$1$$. Since there are no other eigenvalues and eigenvectors, and there is no basis of $$\mathbb{K}^2$$ consisting of eigenvectors of $$A$$, then $$A$$ is not diagonalizable.

Visible text: which has dimension . Since there are no other eigenvalues and eigenvectors, and there is no basis of consisting of eigenvectors of , then is not diagonalizable.

## Requirements for Matrix Diagonalization

If a matrix $$A \in \mathbb{K}^{n \times n}$$ is diagonalizable, then the characteristic polynomial $$\chi_A(t)$$ of $$A$$ over $$\mathbb{K}$$ factors into linear factors:

Visible text: If a matrix is diagonalizable, then the characteristic polynomial of over factors into linear factors:

```math
\chi_A(t) = (\lambda_1 - t) \cdots (\lambda_n - t)
```

where $$A$$ has $$n$$ eigenvalues that need not be distinct $$\lambda_i \in \mathbb{K}$$.

Visible text: where has eigenvalues that need not be distinct .

When all eigenvalues are distinct, the process becomes simpler. If $$A \in \mathbb{K}^{n \times n}$$ and the characteristic polynomial $$\chi_A(t)$$ of $$A$$ over $$\mathbb{K}$$ factors into linear factors:

Visible text: When all eigenvalues are distinct, the process becomes simpler. If and the characteristic polynomial of over factors into linear factors:

```math
\chi_A(t) = (\lambda_1 - t) \cdots (\lambda_n - t)
```

with pairwise distinct eigenvalues $$\lambda_i \neq \lambda_j$$ for $$i \neq j$$ with $$i, j \in \{1, \ldots, n\}$$, then $$A$$ is certainly diagonalizable.

Visible text: with pairwise distinct eigenvalues for with , then is certainly diagonalizable.

Why is this so? Because eigenvectors for pairwise distinct eigenvalues of $$A$$ are always linearly independent and form a basis of $$\mathbb{K}^n$$.

Visible text: Why is this so? Because eigenvectors for pairwise distinct eigenvalues of are always linearly independent and form a basis of .

But what if $$A$$ has repeated eigenvalues? We must check this more carefully. Eigenvalues have algebraic multiplicity $$\mu_A(\lambda_i)$$ and geometric multiplicity $$\dim \text{Eig}_A(\lambda_i)$$ with the relationship:

Visible text: But what if has repeated eigenvalues? We must check this more carefully. Eigenvalues have algebraic multiplicity and geometric multiplicity with the relationship:

```math
\dim \text{Eig}_A(\lambda_i) \leq \mu_A(\lambda_i)
```

## Diagonalization Characterization Theorem

For a matrix $$A \in \mathbb{K}^{n \times n}$$, the following statements are equivalent:

Visible text: For a matrix , the following statements are equivalent:

1. $$A$$ is diagonalizable.

2. Both of the following conditions are satisfied. First, the characteristic polynomial of $$A$$ must factor into linear factors:

   
   
   ```math
   \chi_A(t) = (\lambda_1 - t)^{\mu_A(\lambda_1)} \cdots (\lambda_k - t)^{\mu_A(\lambda_k)}
   ```

   with pairwise distinct eigenvalues $$\lambda_1, \ldots, \lambda_k \in \mathbb{K}$$ of $$A$$. Second, for all eigenvalues of $$A$$, the algebraic multiplicity must equal the geometric multiplicity:

   
   
   ```math
   \mu_A(\lambda_i) = \dim \text{Eig}_A(\lambda_i) \quad \text{for } i = 1, \ldots, k
   ```

3. The direct sum of all eigenspaces is the entire vector space:

   
   
   ```math
   \text{Eig}_A(\lambda_1) \oplus \cdots \oplus \text{Eig}_A(\lambda_k) = \mathbb{K}^n
   ```

   This means there exists a basis of $$\mathbb{K}^n$$ consisting of eigenvectors of $$A$$.

Visible text: 1. is diagonalizable.

2. Both of the following conditions are satisfied. First, the characteristic polynomial of must factor into linear factors:

 
 

 with pairwise distinct eigenvalues of . Second, for all eigenvalues of , the algebraic multiplicity must equal the geometric multiplicity:

 
 

3. The direct sum of all eigenspaces is the entire vector space:

 
 

 This means there exists a basis of consisting of eigenvectors of .

For each $$i = 1, \ldots, k$$, let $$v_1^{(i)}, \ldots, v_{d_i}^{(i)}$$ be a basis of eigenvectors of $$A$$ for the eigenspace $$\text{Eig}_A(\lambda_i)$$. Then:

Visible text: For each , let be a basis of eigenvectors of for the eigenspace . Then:

```math
v_1^{(1)}, \ldots, v_{d_1}^{(1)}, v_1^{(2)}, \ldots, v_{d_2}^{(2)}, \ldots, v_1^{(k)}, \ldots, v_{d_k}^{(k)}
```

is a basis of $$\mathbb{K}^n$$ consisting of eigenvectors of $$A$$. Therefore, $$A$$ is diagonalizable.

Visible text: is a basis of consisting of eigenvectors of . Therefore, is diagonalizable.