# Nakafa Framework: LLM

URL: /en/subject/university/bachelor/ai-ds/linear-methods/matrix-similarity
Source: https://raw.githubusercontent.com/nakafaai/nakafa.com/refs/heads/main/packages/contents/subject/university/bachelor/ai-ds/linear-methods/matrix-similarity/en.mdx

Output docs content for large language models.

---

export const metadata = {
    title: "Matrix Similarity",
    description: "Understand matrix similarity, basis transformations, and invariant properties. Learn eigenvalue preservation, coordinate changes, and linear transformations.",
    authors: [{ name: "Nabil Akbarazzima Fatih" }],
    date: "07/16/2025",
    subject: "Linear Methods of AI",
};

## Definition of Matrix Similarity

In linear algebra, the concept of matrix similarity or equivalence is very important for understanding how two different matrices can represent the same linear transformation in different spaces. Imagine two different portraits of the same object, but taken from different perspectives.

Two matrices <InlineMath math="A, B \in \mathbb{K}^{n \times n}" /> are said to be **similar** if there exists an invertible matrix <InlineMath math="S \in \mathbb{K}^{n \times n}" /> such that:

<BlockMath math="B = S^{-1} \cdot A \cdot S" />

The matrix <InlineMath math="S" /> in this case is called the similarity transformation matrix.

## Basis Transformation and Coordinate Representation

To understand why matrix similarity is so important, we need to look at its relationship with basis transformation. Let <InlineMath math="e_1, \ldots, e_n \in \mathbb{K}^n" /> be the canonical basis and <InlineMath math="v_1, \ldots, v_n \in \mathbb{K}^n" /> be another basis of <InlineMath math="\mathbb{K}^n" />.

If <InlineMath math="S" /> is an invertible matrix with columns <InlineMath math="v_k" />:

<BlockMath math="S = (v_1 \quad \ldots \quad v_n) \in \mathbb{K}^{n \times n}" />

Then we have <InlineMath math="v_k = S \cdot e_k" /> or <InlineMath math="e_k = S^{-1} \cdot v_k" /> for <InlineMath math="k = 1, \ldots, n" />. The matrix <InlineMath math="S" /> represents the **basis transformation**.

A vector <InlineMath math="x \in \mathbb{K}^n" /> can be expressed in the canonical basis through coordinates <InlineMath math="x_k" /> and in the basis <InlineMath math="v_1, \ldots, v_n" /> through coordinates <InlineMath math="\xi_k" />:

<MathContainer>
<BlockMath math="x = \sum_{k=1}^n x_k \cdot e_k" />
<BlockMath math="= \sum_{k=1}^n \xi_k \cdot v_k = S \cdot \xi" />
<BlockMath math="\xi = \begin{pmatrix} \xi_1 \\ \vdots \\ \xi_n \end{pmatrix} = S^{-1} \cdot x" />
</MathContainer>

The matrix <InlineMath math="S^{-1}" /> represents the **coordinate transformation**.

## Linear Transformation in Different Bases

Now consider the linear transformation <InlineMath math="y = A \cdot x" />. In the canonical basis, <InlineMath math="y" /> is expressed through coordinates <InlineMath math="y_k" />, while in the basis <InlineMath math="v_1, \ldots, v_n" /> through coordinates <InlineMath math="\eta_k" />:

<MathContainer>
<BlockMath math="y = \sum_{k=1}^n y_k \cdot e_k" />
<BlockMath math="= \sum_{k=1}^n \eta_k \cdot v_k = S \cdot \eta" />
<BlockMath math="\eta = \begin{pmatrix} \eta_1 \\ \vdots \\ \eta_n \end{pmatrix} = S^{-1} \cdot y" />
</MathContainer>

Therefore:

<MathContainer>
<BlockMath math="S \cdot \eta = y = A \cdot x" />
<BlockMath math="= A \cdot S \cdot \xi" />
</MathContainer>

or in other words:

<BlockMath math="\eta = S^{-1} \cdot A \cdot S \cdot \xi" />

In the basis <InlineMath math="v_1, \ldots, v_n" />, the linear transformation <InlineMath math="y = A \cdot x" /> is represented by <InlineMath math="\eta = B \cdot \xi" /> with the matrix:

<BlockMath math="B = S^{-1} \cdot A \cdot S" />

This is why similar matrices represent the same linear transformation but viewed from different bases. Similar matrices represent the same linear transformation with respect to different bases of <InlineMath math="\mathbb{K}^n" />.

## Invariant Properties of Similar Matrices

Similar matrices have several fundamental properties that are very useful. Since they represent the same linear transformation in different spaces, similar matrices preserve the same intrinsic characteristics.

Based on the theorem about similar matrices, if matrices <InlineMath math="A" /> and <InlineMath math="B = S^{-1} \cdot A \cdot S" /> are similar, then they both have:

1. **The same determinant**
2. **The same characteristic polynomial** 
3. **The same eigenvalues**
4. **The same trace**

### Proof of Determinant Equality

For the determinant, we can show:

<MathContainer>
<BlockMath math="\det(B) = \det(S^{-1} \cdot A \cdot S)" />
<BlockMath math="= \det(S^{-1}) \cdot \det(A) \cdot \det(S)" />
</MathContainer>

Since <InlineMath math="\det(S^{-1}) = \frac{1}{\det(S)}" />, then:

<MathContainer>
<BlockMath math="\det(B) = \frac{1}{\det(S)} \cdot \det(A) \cdot \det(S)" />
<BlockMath math="= \det(A)" />
</MathContainer>

### Eigenvalue Equality

If <InlineMath math="v \in \mathbb{K}^n" /> is an eigenvector of <InlineMath math="A" /> with eigenvalue <InlineMath math="\lambda \in \mathbb{K}" />, such that <InlineMath math="A \cdot v = \lambda \cdot v" />, then <InlineMath math="w = S^{-1} \cdot v" /> is an eigenvector of <InlineMath math="B" /> with the same eigenvalue:

<MathContainer>
<BlockMath math="B \cdot w = S^{-1} \cdot A \cdot S \cdot w" />
<BlockMath math="= S^{-1} \cdot A \cdot v" />
<BlockMath math="= S^{-1} \cdot \lambda \cdot v" />
<BlockMath math="= \lambda \cdot w" />
</MathContainer>

This shows that matrix similarity preserves the spectrum or set of eigenvalues, which is a fundamental characteristic of linear transformations.