Eigenvalues Eigenvectors and Eigenspaces

Eigenvalues and Eigenvectors

Most vectors change both length and direction under a matrix transformation. For an eigenvector, the result stays on the same line. For a real eigenvalue, a positive value preserves its orientation, a negative eigenvalue reverses it, and a zero value sends it to the zero vector.

Let $A \in \mathbb{K}^{n \times n}$ be a square matrix. An $eigenvector$ $v \in \mathbb{K}^n$ for an $eigenvalue$ $\lambda \in \mathbb{K}$ is a non-zero vector $v \neq 0$ that satisfies:

A \cdot v = \lambda \cdot v

For real eigenvalues, multiplying $v$ by $A$ changes its scale or orientation along a real line. Complex eigenvalues describe multiplication by a complex scalar on a one-dimensional complex subspace.

By definition, eigenvalues can equal $0$ , but $eigenvectors are always non-zero$ .

Basic Properties of Eigenvectors

$Scalar Multiplication$ : Let $A \in \mathbb{K}^{n \times n}$ and $v \in \mathbb{K}^n$ with $v \neq 0$ be an eigenvector of $A$ for eigenvalue $\lambda \in \mathbb{K}$ . Then all multiples $t \cdot v$ with $t \neq 0$ are also eigenvectors of $A$ for the same eigenvalue $\lambda$ .

This follows directly from linearity:

\begin{aligned} A \cdot (t \cdot v) &= t \cdot (A \cdot v) \\ &= t \cdot (\lambda \cdot v) \\ &= \lambda \cdot (t \cdot v) \end{aligned}

Because $t\neq0$ , the vector $tv$ is nonzero and satisfies the eigenvector definition for the same eigenvalue.

Examples of Eigenvector Calculations

The examples here $verify the equation$ $Av=\lambda v$ directly, once with a diagonal matrix and once with a symmetric one. Working through both shows where the eigenvalue comes from and how the eigenvector is found, which is the same procedure you need for a matrix that is neither.

Diagonal Matrix

Consider the matrix:

A = \begin{pmatrix} 1 & 0 \\ 0 & 2 \end{pmatrix}

The vector $v_1$ below is an eigenvector for $\lambda_1 = 1$ . $Multiplication by$ $A$ verifies this:

v_1 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}

The vector $v_2$ below is an eigenvector for $\lambda_2 = 2$ . $Multiplication by$ $A$ verifies this:

v_2 = \begin{pmatrix} 0 \\ 1 \end{pmatrix}

A vector outside these coordinate axes generally $changes direction$ . For the diagonal matrix above:

A\begin{pmatrix}1\\1\end{pmatrix}=\begin{pmatrix}1\\2\end{pmatrix}\neq\lambda\begin{pmatrix}1\\1\end{pmatrix}\quad\text{for every }\lambda

Symmetric Matrix

Consider the matrix:

A = \begin{pmatrix} 3 & 1 \\ 1 & 3 \end{pmatrix}

The vector $v_1$ below is an eigenvector for $\lambda_1 = 2$ . $Multiplication$ by $A$ verifies this:

v_1 = \begin{pmatrix} -1 \\ 1 \end{pmatrix}

The vector $v_2$ below is an eigenvector for $\lambda_2 = 4$ . Multiplication by $A$ verifies this:

v_2 = \begin{pmatrix} 1 \\ 1 \end{pmatrix}

Linear Independence of Eigenvectors

Let $A \in \mathbb{K}^{n \times n}$ and $\lambda_1, \ldots, \lambda_k \in \mathbb{K}$ be $pairwise distinct eigenvalues$ of $A$ , so $\lambda_i \neq \lambda_j$ whenever $i \neq j$ . Then corresponding eigenvectors $v_1, \ldots, v_k \in \mathbb{K}^n$ are linearly independent.

By mathematical induction, this theorem implies that an $n \times n$ matrix has at most $n$ distinct eigenvalues.

Eigenspaces and Geometric Multiplicity

For each eigenvalue, we define a vector space containing every solution of $Av=\lambda v$ , including the zero vector.

Let $A \in \mathbb{K}^{n \times n}$ and $\lambda \in \mathbb{K}$ . We call the following set the $eigenspace$ of matrix $A$ for eigenvalue $\lambda$ .

E_A(\lambda) = \{v \in \mathbb{K}^n : A \cdot v = \lambda \cdot v\}

The dimension of an eigenspace is called its $geometric multiplicity$ . We write it as

\dim E_A(\lambda)

The value $\dim E_A(\lambda)$ therefore gives the geometric multiplicity of eigenvalue $\lambda$ for matrix $A$ .

Properties of Eigenspaces

Each eigenspace satisfies the following properties:

$Zero vector is not an eigenvector$ : The eigenspace $E_A(\lambda)$ contains the zero vector along with all eigenvectors for $\lambda$ .
$Set of eigenvectors$ : $E_A(\lambda) \setminus \{0\}$ is the set of all eigenvectors of $A$ corresponding to $\lambda$ .
$Eigenvalue condition$ : $\lambda$ is an eigenvalue of $A$ if and only if $E_A(\lambda) \neq \{0\}$ .
$Dimension bound$ : $0 \leq \dim E_A(\lambda) \leq n$ . If $\lambda$ is an eigenvalue, the lower bound improves to $1$ .
$Relationship with kernel$ : $E_A(0) = \{v \in \mathbb{K}^n : A \cdot v = 0\} = \ker A$ .
$General eigenspace$ : $E_A(\lambda) = \{v \in \mathbb{K}^n : (A - \lambda \cdot I) \cdot v = 0\} = \ker(A - \lambda \cdot I)$ .
$Intersection of eigenspaces$ : If $\lambda_1 \neq \lambda_2$ , then $E_A(\lambda_1) \cap E_A(\lambda_2) = \{0\}$ .

Relationship with Invertibility

Matrix $A \in \mathbb{K}^{n \times n}$ is invertible if and only if $zero is not one$ of its eigenvalues.

Indeed, $A$ is invertible if and only if $\operatorname{rank} A = n$ , equivalently $E_A(0) = \ker A = \{0\}$ . An eigenvalue of zero therefore means that the matrix sends some nonzero vector to zero, and that is exactly what rules out invertibility.

Eigenvalues of Inverse Matrix

If matrix $A$ is invertible and $v \neq 0$ is an eigenvector of $A$ for eigenvalue $\lambda \in \mathbb{K}$ , then $v$ is also an eigenvector of $A^{-1}$ for eigenvalue $\frac{1}{\lambda}$ .

$Applying$ $A^{-1}$ to $A \cdot v = \lambda \cdot v$ gives:

v = \lambda A^{-1}v \quad \Longrightarrow \quad A^{-1}v = \frac{1}{\lambda}v

Invertibility Criteria

For a square matrix $A \in \mathbb{K}^{n \times n}$ , the following statements are equivalent:

$A$ is invertible
There exists a matrix $A^{-1} \in \mathbb{K}^{n \times n}$ with $A \cdot A^{-1} = I = A^{-1} \cdot A$
$A$ has full rank, $\operatorname{rank} A = n$ , equivalently $\ker A = \{0\}$
The columns of $A$ are linearly independent
The rows of $A$ are linearly independent
$\det A \neq 0$
$0$ is not an eigenvalue of $A$

$Any one of these conditions$ is enough to prove that the matrix is invertible.

Eigenvalues and Eigenvectors

Let $A \in \mathbb{K}^{n \times n}$ be a square matrix. An $eigenvector$ $v \in \mathbb{K}^n$ for an $eigenvalue$ $\lambda \in \mathbb{K}$ is a non-zero vector $v \neq 0$ that satisfies:

A \cdot v = \lambda \cdot v

For real eigenvalues, multiplying $v$ by $A$ changes its scale or orientation along a real line. Complex eigenvalues describe multiplication by a complex scalar on a one-dimensional complex subspace.

By definition, eigenvalues can equal $0$ , but $eigenvectors are always non-zero$ .

Basic Properties of Eigenvectors

This follows directly from linearity:

\begin{aligned} A \cdot (t \cdot v) &= t \cdot (A \cdot v) \\ &= t \cdot (\lambda \cdot v) \\ &= \lambda \cdot (t \cdot v) \end{aligned}

Because $t\neq0$ , the vector $tv$ is nonzero and satisfies the eigenvector definition for the same eigenvalue.

Examples of Eigenvector Calculations

Diagonal Matrix

Consider the matrix:

A = \begin{pmatrix} 1 & 0 \\ 0 & 2 \end{pmatrix}

The vector $v_1$ below is an eigenvector for $\lambda_1 = 1$ . $Multiplication by$ $A$ verifies this:

v_1 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}

The vector $v_2$ below is an eigenvector for $\lambda_2 = 2$ . $Multiplication by$ $A$ verifies this:

v_2 = \begin{pmatrix} 0 \\ 1 \end{pmatrix}

A vector outside these coordinate axes generally $changes direction$ . For the diagonal matrix above:

A\begin{pmatrix}1\\1\end{pmatrix}=\begin{pmatrix}1\\2\end{pmatrix}\neq\lambda\begin{pmatrix}1\\1\end{pmatrix}\quad\text{for every }\lambda

Symmetric Matrix

Consider the matrix:

A = \begin{pmatrix} 3 & 1 \\ 1 & 3 \end{pmatrix}

The vector $v_1$ below is an eigenvector for $\lambda_1 = 2$ . $Multiplication$ by $A$ verifies this:

v_1 = \begin{pmatrix} -1 \\ 1 \end{pmatrix}

The vector $v_2$ below is an eigenvector for $\lambda_2 = 4$ . Multiplication by $A$ verifies this:

v_2 = \begin{pmatrix} 1 \\ 1 \end{pmatrix}

Linear Independence of Eigenvectors

By mathematical induction, this theorem implies that an $n \times n$ matrix has at most $n$ distinct eigenvalues.

Eigenspaces and Geometric Multiplicity

For each eigenvalue, we define a vector space containing every solution of $Av=\lambda v$ , including the zero vector.

Let $A \in \mathbb{K}^{n \times n}$ and $\lambda \in \mathbb{K}$ . We call the following set the $eigenspace$ of matrix $A$ for eigenvalue $\lambda$ .

E_A(\lambda) = \{v \in \mathbb{K}^n : A \cdot v = \lambda \cdot v\}

The dimension of an eigenspace is called its $geometric multiplicity$ . We write it as

\dim E_A(\lambda)

The value $\dim E_A(\lambda)$ therefore gives the geometric multiplicity of eigenvalue $\lambda$ for matrix $A$ .

Properties of Eigenspaces

Each eigenspace satisfies the following properties:

$Zero vector is not an eigenvector$ : The eigenspace $E_A(\lambda)$ contains the zero vector along with all eigenvectors for $\lambda$ .
$Set of eigenvectors$ : $E_A(\lambda) \setminus \{0\}$ is the set of all eigenvectors of $A$ corresponding to $\lambda$ .
$Eigenvalue condition$ : $\lambda$ is an eigenvalue of $A$ if and only if $E_A(\lambda) \neq \{0\}$ .
$Dimension bound$ : $0 \leq \dim E_A(\lambda) \leq n$ . If $\lambda$ is an eigenvalue, the lower bound improves to $1$ .
$Relationship with kernel$ : $E_A(0) = \{v \in \mathbb{K}^n : A \cdot v = 0\} = \ker A$ .
$General eigenspace$ : $E_A(\lambda) = \{v \in \mathbb{K}^n : (A - \lambda \cdot I) \cdot v = 0\} = \ker(A - \lambda \cdot I)$ .
$Intersection of eigenspaces$ : If $\lambda_1 \neq \lambda_2$ , then $E_A(\lambda_1) \cap E_A(\lambda_2) = \{0\}$ .

Relationship with Invertibility

Matrix $A \in \mathbb{K}^{n \times n}$ is invertible if and only if $zero is not one$ of its eigenvalues.

Eigenvalues of Inverse Matrix

If matrix $A$ is invertible and $v \neq 0$ is an eigenvector of $A$ for eigenvalue $\lambda \in \mathbb{K}$ , then $v$ is also an eigenvector of $A^{-1}$ for eigenvalue $\frac{1}{\lambda}$ .

$Applying$ $A^{-1}$ to $A \cdot v = \lambda \cdot v$ gives:

v = \lambda A^{-1}v \quad \Longrightarrow \quad A^{-1}v = \frac{1}{\lambda}v

Invertibility Criteria

For a square matrix $A \in \mathbb{K}^{n \times n}$ , the following statements are equivalent:

$A$ is invertible
There exists a matrix $A^{-1} \in \mathbb{K}^{n \times n}$ with $A \cdot A^{-1} = I = A^{-1} \cdot A$
$A$ has full rank, $\operatorname{rank} A = n$ , equivalently $\ker A = \{0\}$
The columns of $A$ are linearly independent
The rows of $A$ are linearly independent
$\det A \neq 0$
$0$ is not an eigenvalue of $A$

$Any one of these conditions$ is enough to prove that the matrix is invertible.