# Nakafa Learning Content

> For AI agents: use [llms.txt](https://nakafa.com/llms.txt) for the site index. Markdown versions are available by appending `.md` to content URLs or sending `Accept: text/markdown`.

URL: https://nakafa.com/en/subjects/ai-ds/linear-methods/eigenvalue-eigenvector-eigenspace
Source: https://raw.githubusercontent.com/nakafaai/nakafa.com/refs/heads/main/packages/contents/material/lesson/ai-ds/linear-methods/eigenvalue-eigenvector-eigenspace/en.mdx

Learn fundamental eigenvalue concepts: understand how matrices transform vectors, calculate eigenvectors, and explore eigenspace properties.

---

## Definition of Fundamental Concepts

In linear algebra, we are often interested in special vectors that have unique properties when multiplied by matrices. Imagine vectors that are only "stretched" or "shortened" by the matrix, but their direction remains unchanged.

Let $$A \in \mathbb{K}^{n \times n}$$ be a square matrix. An **eigenvector** $$v \in \mathbb{K}^n$$ for an **eigenvalue** $$\lambda \in \mathbb{K}$$ is a non-zero vector $$v \neq 0$$ that satisfies:

Visible text: Let be a square matrix. An **eigenvector** for an **eigenvalue** is a non-zero vector that satisfies:

```math
A \cdot v = \lambda \cdot v
```

This equation shows that when matrix $$A$$ operates on vector $$v$$, the result is a scalar multiple of the same vector.

Visible text: This equation shows that when matrix operates on vector , the result is a scalar multiple of the same vector.

> By definition, eigenvalues can equal $$0$$, but eigenvectors are always non-zero.

Visible text: > By definition, eigenvalues can equal , but eigenvectors are always non-zero.

## Basic Properties of Eigenvectors

Eigenvectors have fundamental properties that are very useful in various mathematical applications.

**Scalar Multiplication**: Let $$A \in \mathbb{K}^{n \times n}$$ and $$v \in \mathbb{K}^n$$ with $$v \neq 0$$ be an eigenvector of $$A$$ for eigenvalue $$\lambda \in \mathbb{K}$$. Then all multiples $$t \cdot v$$ with $$t \neq 0$$ are also eigenvectors of $$A$$ for the same eigenvalue $$\lambda$$.

Visible text: **Scalar Multiplication**: Let and with be an eigenvector of for eigenvalue . Then all multiples with are also eigenvectors of for the same eigenvalue .

Why is this true? $$A \cdot (t \cdot v) = t \cdot (A \cdot v) = t \cdot (\lambda \cdot v) = \lambda \cdot (t \cdot v)$$

Visible text: Why is this true?

This property shows that if we find one eigenvector, then all its non-zero multiples are also eigenvectors for the same eigenvalue.

## Examples of Eigenvector Calculations

Let's look at some concrete examples to better understand this concept:

### Diagonal Matrix

For matrix $$A = \begin{pmatrix} 1 & 0 \\ 0 & 2 \end{pmatrix}$$:

Visible text: For matrix :

$$v_1 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}$$ is an eigenvector for eigenvalue $$\lambda_1 = 1$$, because $$A \cdot v_1 = \begin{pmatrix} 1 \\ 0 \end{pmatrix} = 1 \cdot v_1$$

Visible text: is an eigenvector for eigenvalue , because

$$v_2 = \begin{pmatrix} 0 \\ 1 \end{pmatrix}$$ is an eigenvector for eigenvalue $$\lambda_2 = 2$$, because $$A \cdot v_2 = \begin{pmatrix} 0 \\ 2 \end{pmatrix} = 2 \cdot v_2$$

Visible text: is an eigenvector for eigenvalue , because

### Symmetric Matrix

For matrix $$A = \begin{pmatrix} 3 & 1 \\ 1 & 3 \end{pmatrix}$$:

Visible text: For matrix :

$$v_1 = \begin{pmatrix} -1 \\ 1 \end{pmatrix}$$ is an eigenvector for eigenvalue $$\lambda_1 = 2$$, because $$A \cdot v_1 = \begin{pmatrix} -2 \\ 2 \end{pmatrix} = 2 \cdot v_1$$

Visible text: is an eigenvector for eigenvalue , because

$$v_2 = \begin{pmatrix} 1 \\ 1 \end{pmatrix}$$ is an eigenvector for eigenvalue $$\lambda_2 = 4$$, because $$A \cdot v_2 = \begin{pmatrix} 4 \\ 4 \end{pmatrix} = 4 \cdot v_2$$

Visible text: is an eigenvector for eigenvalue , because

## Linear Independence of Eigenvectors

One important result in eigenvector theory is that eigenvectors corresponding to different eigenvalues are always linearly independent.

There's a very important result about linear independence of eigenvectors. Let $$A \in \mathbb{K}^{n \times n}$$ and $$\lambda_1, \ldots, \lambda_k \in \mathbb{K}$$ be pairwise distinct eigenvalues of $$A$$, that is $$\lambda_i \neq \lambda_j$$ for $$i \neq j$$ with $$i, j \in \{1, \ldots, k\}$$. Then the corresponding eigenvectors $$v_1, \ldots, v_k \in \mathbb{K}^n$$ are linearly independent.

Visible text: There's a very important result about linear independence of eigenvectors. Let and be pairwise distinct eigenvalues of , that is for with . Then the corresponding eigenvectors are linearly independent.

This theorem can be proven using mathematical induction and has the important consequence that an $$n \times n$$ matrix has at most $$n$$ distinct eigenvalues.

Visible text: This theorem can be proven using mathematical induction and has the important consequence that an matrix has at most distinct eigenvalues.

## Eigenspaces and Geometric Multiplicity

For each eigenvalue, we can define a vector space consisting of all eigenvectors corresponding to that eigenvalue.

Let $$A \in \mathbb{K}^{n \times n}$$ and $$\lambda \in \mathbb{K}$$. The set:

Visible text: Let and . The set:

```math
\text{Eig}_A(\lambda) = \{v \in \mathbb{K}^n : A \cdot v = \lambda \cdot v\}
```

is called the **eigenspace** of $$A$$ for eigenvalue $$\lambda$$. The dimension of $$\text{Eig}_A(\lambda)$$:

Visible text: is called the **eigenspace** of for eigenvalue . The dimension of :

```math
\dim \text{Eig}_A(\lambda)
```

is called the **geometric multiplicity** of eigenvalue $$\lambda$$ of $$A$$.

Visible text: is called the **geometric multiplicity** of eigenvalue of .

### Properties of Eigenspaces

Eigenspaces have several important properties:

1. **Zero vector is not an eigenvector**: The zero vector is not an eigenvector, but it is an element of $$\text{Eig}_A(\lambda)$$

2. **Set of eigenvectors**: $$\text{Eig}_A(\lambda) \setminus \{0\}$$ is the set of all eigenvectors of $$A$$ corresponding to $$\lambda$$

3. **Eigenvalue condition**: $$\lambda$$ is an eigenvalue of $$A$$ if and only if $$\text{Eig}_A(\lambda) \neq \{0\}$$

4. **Dimension bound**: $$0 \leq \dim \text{Eig}_A(\lambda) \leq n$$

5. **Relationship with kernel**: $$\text{Eig}_A(0) = \{v \in \mathbb{K}^n : A \cdot v = 0\} = \ker A$$

6. **General eigenspace**: $$\text{Eig}_A(\lambda) = \{v \in \mathbb{K}^n : (A - \lambda \cdot I) \cdot v = 0\} = \ker(A - \lambda \cdot I)$$

7. **Intersection of eigenspaces**: If $$\lambda_1 \neq \lambda_2$$, then $$\text{Eig}_A(\lambda_1) \cap \text{Eig}_A(\lambda_2) = \{0\}$$

Visible text: 1. **Zero vector is not an eigenvector**: The zero vector is not an eigenvector, but it is an element of 

2. **Set of eigenvectors**: is the set of all eigenvectors of corresponding to 

3. **Eigenvalue condition**: is an eigenvalue of if and only if 

4. **Dimension bound**: 

5. **Relationship with kernel**: 

6. **General eigenspace**: 

7. **Intersection of eigenspaces**: If , then

## Relationship with Invertibility

Eigenvalues have a close relationship with the invertibility property of matrices.

Now, let's look at an interesting relationship between invertibility and eigenvalues. Matrix $$A \in \mathbb{K}^{n \times n}$$ is invertible if and only if all eigenvalues $$\lambda \in \mathbb{K}$$ of $$A$$ satisfy $$\lambda \neq 0$$.

Visible text: Now, let's look at an interesting relationship between invertibility and eigenvalues. Matrix is invertible if and only if all eigenvalues of satisfy .

Why is this true? $$A$$ is invertible if and only if $$\text{Rank} A = n$$, which means $$\text{Eig}_A(0) = \ker A = \{0\}$$, so $$\lambda = 0$$ is not an eigenvalue of $$A$$.

Visible text: Why is this true? is invertible if and only if , which means , so is not an eigenvalue of .

### Eigenvalues of Inverse Matrix

If matrix $$A$$ is invertible and $$v \neq 0$$ is an eigenvector of $$A$$ for eigenvalue $$\lambda \in \mathbb{K}$$, then $$v$$ is also an eigenvector of $$A^{-1}$$ for eigenvalue $$\frac{1}{\lambda}$$.

Visible text: If matrix is invertible and is an eigenvector of for eigenvalue , then is also an eigenvector of for eigenvalue .

Why is this true? From $$A \cdot v = \lambda \cdot v$$, by multiplying $$A^{-1}$$ and $$\frac{1}{\lambda}$$, we get $$\frac{1}{\lambda} \cdot v = A^{-1} \cdot v$$.

Visible text: Why is this true? From , by multiplying and , we get .

## Invertibility Criteria

Let's look at various ways to determine whether a matrix is invertible. For a square matrix $$A \in \mathbb{K}^{n \times n}$$, the following statements are equivalent:

Visible text: Let's look at various ways to determine whether a matrix is invertible. For a square matrix , the following statements are equivalent:

1. $$A$$ is invertible
2. There exists a matrix $$A^{-1} \in \mathbb{K}^{n \times n}$$ with $$A \cdot A^{-1} = I = A^{-1} \cdot A$$
3. $$A$$ has full rank, $$\text{Rank} A = n$$ or $$\ker A = \{0\}$$
4. The columns of $$A$$ are linearly independent
5. The rows of $$A$$ are linearly independent
6. $$\det A \neq 0$$
7. All eigenvalues of $$A$$ are not equal to $$0$$

Visible text: 1. is invertible
2. There exists a matrix with 
3. has full rank, or 
4. The columns of are linearly independent
5. The rows of are linearly independent
6. 
7. All eigenvalues of are not equal to

This theorem provides various equivalent ways to check whether a matrix is invertible, with eigenvalues being one of the very useful criteria.