Eigenvalues, Eigenvectors, and Eigenspaces

Definition of Fundamental Concepts

In linear algebra, we are often interested in special vectors that have unique properties when multiplied by matrices. Imagine vectors that are only "stretched" or "shortened" by the matrix, but their direction remains unchanged.

Let $A \in \mathbb{K}^{n \times n}$ be a square matrix. An eigenvector $v \in \mathbb{K}^n$ for an eigenvalue $\lambda \in \mathbb{K}$ is a non-zero vector $v \neq 0$ that satisfies:

This equation shows that when matrix $A$ operates on vector $v$, the result is a scalar multiple of the same vector.

By definition, eigenvalues can equal $0$, but eigenvectors are always non-zero.

Basic Properties of Eigenvectors

Eigenvectors have fundamental properties that are very useful in various mathematical applications.

Scalar Multiplication: Let $A \in \mathbb{K}^{n \times n}$ and $v \in \mathbb{K}^n$ with $v \neq 0$ be an eigenvector of $A$ for eigenvalue $\lambda \in \mathbb{K}$. Then all multiples $t \cdot v$ with $t \neq 0$ are also eigenvectors of $A$ for the same eigenvalue $\lambda$.

Why is this true? $A \cdot (t \cdot v) = t \cdot (A \cdot v) = t \cdot (\lambda \cdot v) = \lambda \cdot (t \cdot v)$

This property shows that if we find one eigenvector, then all its non-zero multiples are also eigenvectors for the same eigenvalue.

Examples of Eigenvector Calculations

Let's look at some concrete examples to better understand this concept:

Diagonal Matrix

For matrix $A = \begin{pmatrix} 1 & 0 \\ 0 & 2 \end{pmatrix}$:

$v_1 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}$ is an eigenvector for eigenvalue $\lambda_1 = 1$, because $A \cdot v_1 = \begin{pmatrix} 1 \\ 0 \end{pmatrix} = 1 \cdot v_1$

$v_2 = \begin{pmatrix} 0 \\ 1 \end{pmatrix}$ is an eigenvector for eigenvalue $\lambda_2 = 2$, because $A \cdot v_2 = \begin{pmatrix} 0 \\ 2 \end{pmatrix} = 2 \cdot v_2$

Symmetric Matrix

For matrix $A = \begin{pmatrix} 3 & 1 \\ 1 & 3 \end{pmatrix}$:

$v_1 = \begin{pmatrix} -1 \\ 1 \end{pmatrix}$ is an eigenvector for eigenvalue $\lambda_1 = 2$, because $A \cdot v_1 = \begin{pmatrix} -2 \\ 2 \end{pmatrix} = 2 \cdot v_1$

$v_2 = \begin{pmatrix} 1 \\ 1 \end{pmatrix}$ is an eigenvector for eigenvalue $\lambda_2 = 4$, because $A \cdot v_2 = \begin{pmatrix} 4 \\ 4 \end{pmatrix} = 4 \cdot v_2$

Linear Independence of Eigenvectors

One important result in eigenvector theory is that eigenvectors corresponding to different eigenvalues are always linearly independent.

There's a very important result about linear independence of eigenvectors. Let $A \in \mathbb{K}^{n \times n}$ and $\lambda_1, \ldots, \lambda_k \in \mathbb{K}$ be pairwise distinct eigenvalues of $A$, that is $\lambda_i \neq \lambda_j$ for $i \neq j$ with $i, j \in \{1, \ldots, k\}$. Then the corresponding eigenvectors $v_1, \ldots, v_k \in \mathbb{K}^n$ are linearly independent.

This theorem can be proven using mathematical induction and has the important consequence that an $n \times n$ matrix has at most $n$ distinct eigenvalues.

Eigenspaces and Geometric Multiplicity

For each eigenvalue, we can define a vector space consisting of all eigenvectors corresponding to that eigenvalue.

Let $A \in \mathbb{K}^{n \times n}$ and $\lambda \in \mathbb{K}$. The set:

is called the eigenspace of $A$ for eigenvalue $\lambda$. The dimension of $\text{Eig}_A(\lambda)$:

is called the geometric multiplicity of eigenvalue $\lambda$ of $A$.

Properties of Eigenspaces

Eigenspaces have several important properties:

1. Zero vector is not an eigenvector: The zero vector is not an eigenvector, but it is an element of $\text{Eig}_A(\lambda)$

2. Set of eigenvectors: $\text{Eig}_A(\lambda) \setminus \{0\}$ is the set of all eigenvectors of $A$ corresponding to $\lambda$

3. Eigenvalue condition: $\lambda$ is an eigenvalue of $A$ if and only if $\text{Eig}_A(\lambda) \neq \{0\}$

4. Dimension bound: $0 \leq \dim \text{Eig}_A(\lambda) \leq n$

5. Relationship with kernel: $\text{Eig}_A(0) = \{v \in \mathbb{K}^n : A \cdot v = 0\} = \ker A$

6. General eigenspace: $\text{Eig}_A(\lambda) = \{v \in \mathbb{K}^n : (A - \lambda \cdot I) \cdot v = 0\} = \ker(A - \lambda \cdot I)$

7. Intersection of eigenspaces: If $\lambda_1 \neq \lambda_2$, then $\text{Eig}_A(\lambda_1) \cap \text{Eig}_A(\lambda_2) = \{0\}$

Relationship with Invertibility

Eigenvalues have a close relationship with the invertibility property of matrices.

Now, let's look at an interesting relationship between invertibility and eigenvalues. Matrix $A \in \mathbb{K}^{n \times n}$ is invertible if and only if all eigenvalues $\lambda \in \mathbb{K}$ of $A$ satisfy $\lambda \neq 0$.

Why is this true? $A$ is invertible if and only if $\text{Rank} A = n$, which means $\text{Eig}_A(0) = \ker A = \{0\}$, so $\lambda = 0$ is not an eigenvalue of $A$.

Eigenvalues of Inverse Matrix

If matrix $A$ is invertible and $v \neq 0$ is an eigenvector of $A$ for eigenvalue $\lambda \in \mathbb{K}$, then $v$ is also an eigenvector of $A^{-1}$ for eigenvalue $\frac{1}{\lambda}$.

Why is this true? From $A \cdot v = \lambda \cdot v$, by multiplying $A^{-1}$ and $\frac{1}{\lambda}$, we get $\frac{1}{\lambda} \cdot v = A^{-1} \cdot v$.

Invertibility Criteria

Let's look at various ways to determine whether a matrix is invertible. For a square matrix $A \in \mathbb{K}^{n \times n}$, the following statements are equivalent:

1. $A$ is invertible
2. There exists a matrix $A^{-1} \in \mathbb{K}^{n \times n}$ with $A \cdot A^{-1} = I = A^{-1} \cdot A$
3. $A$ has full rank, $\text{Rank} A = n$ or $\ker A = \{0\}$
4. The columns of $A$ are linearly independent
5. The rows of $A$ are linearly independent
6. $\det A \neq 0$
7. All eigenvalues of $A$ are not equal to $0$

This theorem provides various equivalent ways to check whether a matrix is invertible, with eigenvalues being one of the very useful criteria.