# Nakafa Framework: LLM

URL: /en/subject/university/bachelor/ai-ds/linear-methods/eigenvalue-eigenvector-eigenspace
Source: https://raw.githubusercontent.com/nakafaai/nakafa.com/refs/heads/main/packages/contents/subject/university/bachelor/ai-ds/linear-methods/eigenvalue-eigenvector-eigenspace/en.mdx

Output docs content for large language models.

---

export const metadata = {
    title: "Eigenvalues, Eigenvectors, and Eigenspaces",
    description: "Master fundamental eigenvalue concepts: understand how matrices transform vectors, calculate eigenvectors, and explore eigenspace properties.",
    authors: [{ name: "Nabil Akbarazzima Fatih" }],
    date: "07/12/2025",
    subject: "Linear Methods of AI",
};

## Definition of Fundamental Concepts

In linear algebra, we are often interested in special vectors that have unique properties when multiplied by matrices. Imagine vectors that are only "stretched" or "shortened" by the matrix, but their direction remains unchanged.

Let <InlineMath math="A \in \mathbb{K}^{n \times n}" /> be a square matrix. An **eigenvector** <InlineMath math="v \in \mathbb{K}^n" /> for an **eigenvalue** <InlineMath math="\lambda \in \mathbb{K}" /> is a non-zero vector <InlineMath math="v \neq 0" /> that satisfies:

<BlockMath math="A \cdot v = \lambda \cdot v" />

This equation shows that when matrix <InlineMath math="A" /> operates on vector <InlineMath math="v" />, the result is a scalar multiple of the same vector.

> By definition, eigenvalues can equal 0, but eigenvectors are always non-zero.

## Basic Properties of Eigenvectors

Eigenvectors have fundamental properties that are very useful in various mathematical applications.

**Scalar Multiplication**: Let <InlineMath math="A \in \mathbb{K}^{n \times n}" /> and <InlineMath math="v \in \mathbb{K}^n" /> with <InlineMath math="v \neq 0" /> be an eigenvector of <InlineMath math="A" /> for eigenvalue <InlineMath math="\lambda \in \mathbb{K}" />. Then all multiples <InlineMath math="t \cdot v" /> with <InlineMath math="t \neq 0" /> are also eigenvectors of <InlineMath math="A" /> for the same eigenvalue <InlineMath math="\lambda" />.

Why is this true? <InlineMath math="A \cdot (t \cdot v) = t \cdot (A \cdot v) = t \cdot (\lambda \cdot v) = \lambda \cdot (t \cdot v)" />

This property shows that if we find one eigenvector, then all its non-zero multiples are also eigenvectors for the same eigenvalue.

## Examples of Eigenvector Calculations

Let's look at some concrete examples to better understand this concept:

### Diagonal Matrix

For matrix <InlineMath math="A = \begin{pmatrix} 1 & 0 \\ 0 & 2 \end{pmatrix}" />:

<InlineMath math="v_1 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}" /> is an eigenvector for eigenvalue <InlineMath math="\lambda_1 = 1" />, because <InlineMath math="A \cdot v_1 = \begin{pmatrix} 1 \\ 0 \end{pmatrix} = 1 \cdot v_1" />

<InlineMath math="v_2 = \begin{pmatrix} 0 \\ 1 \end{pmatrix}" /> is an eigenvector for eigenvalue <InlineMath math="\lambda_2 = 2" />, because <InlineMath math="A \cdot v_2 = \begin{pmatrix} 0 \\ 2 \end{pmatrix} = 2 \cdot v_2" />

### Symmetric Matrix

For matrix <InlineMath math="A = \begin{pmatrix} 3 & 1 \\ 1 & 3 \end{pmatrix}" />:

<InlineMath math="v_1 = \begin{pmatrix} -1 \\ 1 \end{pmatrix}" /> is an eigenvector for eigenvalue <InlineMath math="\lambda_1 = 2" />, because <InlineMath math="A \cdot v_1 = \begin{pmatrix} -2 \\ 2 \end{pmatrix} = 2 \cdot v_1" />

<InlineMath math="v_2 = \begin{pmatrix} 1 \\ 1 \end{pmatrix}" /> is an eigenvector for eigenvalue <InlineMath math="\lambda_2 = 4" />, because <InlineMath math="A \cdot v_2 = \begin{pmatrix} 4 \\ 4 \end{pmatrix} = 4 \cdot v_2" />

## Linear Independence of Eigenvectors

One important result in eigenvector theory is that eigenvectors corresponding to different eigenvalues are always linearly independent.

There's a very important result about linear independence of eigenvectors. Let <InlineMath math="A \in \mathbb{K}^{n \times n}" /> and <InlineMath math="\lambda_1, \ldots, \lambda_k \in \mathbb{K}" /> be pairwise distinct eigenvalues of <InlineMath math="A" />, that is <InlineMath math="\lambda_i \neq \lambda_j" /> for <InlineMath math="i \neq j" /> with <InlineMath math="i, j \in \{1, \ldots, k\}" />. Then the corresponding eigenvectors <InlineMath math="v_1, \ldots, v_k \in \mathbb{K}^n" /> are linearly independent.

This theorem can be proven using mathematical induction and has the important consequence that an <InlineMath math="n \times n" /> matrix has at most <InlineMath math="n" /> distinct eigenvalues.

## Eigenspaces and Geometric Multiplicity

For each eigenvalue, we can define a vector space consisting of all eigenvectors corresponding to that eigenvalue.

Let <InlineMath math="A \in \mathbb{K}^{n \times n}" /> and <InlineMath math="\lambda \in \mathbb{K}" />. The set:

<BlockMath math="\text{Eig}_A(\lambda) = \{v \in \mathbb{K}^n : A \cdot v = \lambda \cdot v\}" />

is called the **eigenspace** of <InlineMath math="A" /> for eigenvalue <InlineMath math="\lambda" />. The dimension of <InlineMath math="\text{Eig}_A(\lambda)" />:

<BlockMath math="\dim \text{Eig}_A(\lambda)" />

is called the **geometric multiplicity** of eigenvalue <InlineMath math="\lambda" /> of <InlineMath math="A" />.

### Properties of Eigenspaces

Eigenspaces have several important properties:

1. **Zero vector is not an eigenvector**: The zero vector is not an eigenvector, but it is an element of <InlineMath math="\text{Eig}_A(\lambda)" />

2. **Set of eigenvectors**: <InlineMath math="\text{Eig}_A(\lambda) \setminus \{0\}" /> is the set of all eigenvectors of <InlineMath math="A" /> corresponding to <InlineMath math="\lambda" />

3. **Eigenvalue condition**: <InlineMath math="\lambda" /> is an eigenvalue of <InlineMath math="A" /> if and only if <InlineMath math="\text{Eig}_A(\lambda) \neq \{0\}" />

4. **Dimension bound**: <InlineMath math="0 \leq \dim \text{Eig}_A(\lambda) \leq n" />

5. **Relationship with kernel**: <InlineMath math="\text{Eig}_A(0) = \{v \in \mathbb{K}^n : A \cdot v = 0\} = \ker A" />

6. **General eigenspace**: <InlineMath math="\text{Eig}_A(\lambda) = \{v \in \mathbb{K}^n : (A - \lambda \cdot I) \cdot v = 0\} = \ker(A - \lambda \cdot I)" />

7. **Intersection of eigenspaces**: If <InlineMath math="\lambda_1 \neq \lambda_2" />, then <InlineMath math="\text{Eig}_A(\lambda_1) \cap \text{Eig}_A(\lambda_2) = \{0\}" />

## Relationship with Invertibility

Eigenvalues have a close relationship with the invertibility property of matrices.

Now, let's look at an interesting relationship between invertibility and eigenvalues. Matrix <InlineMath math="A \in \mathbb{K}^{n \times n}" /> is invertible if and only if all eigenvalues <InlineMath math="\lambda \in \mathbb{K}" /> of <InlineMath math="A" /> satisfy <InlineMath math="\lambda \neq 0" />.

Why is this true? <InlineMath math="A" /> is invertible if and only if <InlineMath math="\text{Rank} A = n" />, which means <InlineMath math="\text{Eig}_A(0) = \ker A = \{0\}" />, so <InlineMath math="\lambda = 0" /> is not an eigenvalue of <InlineMath math="A" />.

### Eigenvalues of Inverse Matrix

If matrix <InlineMath math="A" /> is invertible and <InlineMath math="v \neq 0" /> is an eigenvector of <InlineMath math="A" /> for eigenvalue <InlineMath math="\lambda \in \mathbb{K}" />, then <InlineMath math="v" /> is also an eigenvector of <InlineMath math="A^{-1}" /> for eigenvalue <InlineMath math="\frac{1}{\lambda}" />.

Why is this true? From <InlineMath math="A \cdot v = \lambda \cdot v" />, by multiplying <InlineMath math="A^{-1}" /> and <InlineMath math="\frac{1}{\lambda}" />, we get <InlineMath math="\frac{1}{\lambda} \cdot v = A^{-1} \cdot v" />.

## Invertibility Criteria

Let's look at various ways to determine whether a matrix is invertible. For a square matrix <InlineMath math="A \in \mathbb{K}^{n \times n}" />, the following statements are equivalent:

1. <InlineMath math="A" /> is invertible
2. There exists a matrix <InlineMath math="A^{-1} \in \mathbb{K}^{n \times n}" /> with <InlineMath math="A \cdot A^{-1} = I = A^{-1} \cdot A" />
3. <InlineMath math="A" /> has full rank, <InlineMath math="\text{Rank} A = n" /> or <InlineMath math="\ker A = \{0\}" />
4. The columns of <InlineMath math="A" /> are linearly independent
5. The rows of <InlineMath math="A" /> are linearly independent
6. <InlineMath math="\det A \neq 0" />
7. All eigenvalues of <InlineMath math="A" /> are not equal to 0

This theorem provides various equivalent ways to check whether a matrix is invertible, with eigenvalues being one of the very useful criteria.