# Nakafa Learning Content > For AI agents: use [llms.txt](https://nakafa.com/llms.txt) for the site index. Markdown versions are available by appending `.md` to content URLs or sending `Accept: text/markdown`. URL: https://nakafa.com/en/subjects/ai-ds/linear-methods/spectral-complex-matrix Source: https://raw.githubusercontent.com/nakafaai/nakafa.com/refs/heads/main/packages/contents/material/lesson/ai-ds/linear-methods/spectral-complex-matrix/en.mdx Learn orthogonal diagonalization of normal matrices: prove equivalence between normality and eigenvector bases using mathematical induction. --- ## Orthogonal Diagonalization Characterization The spectral theorem for complex matrices provides a complete characterization of when a complex matrix can be diagonalized using an orthonormal basis of eigenvectors. This is a very important result in linear algebra because it connects geometric concepts (orthonormal basis) with algebraic concepts (matrix normality). For a complex matrix $$A \in \mathbb{C}^{n \times n}$$, the following conditions are equivalent to each other: Visible text: For a complex matrix , the following conditions are equivalent to each other: 1. There exists an orthonormal basis of $$\mathbb{C}^n$$ consisting of eigenvectors of matrix $$A$$ 2. Matrix $$A$$ is normal Visible text: 1. There exists an orthonormal basis of consisting of eigenvectors of matrix 2. Matrix is normal This equivalence is important because it shows that algebraic properties (normality) are directly related to the possibility of orthogonal diagonalization. ## Proof of the First Direction Let us prove that if there exists an orthonormal basis of eigenvectors, then the matrix is normal. This is like proving that if a building has a very regular and symmetric structure, then the building has special balance properties. Let $$v_1, \ldots, v_n$$ be an orthonormal basis consisting of eigenvectors of matrix $$A$$. For each $$j = 1, \ldots, n$$ we have $$Av_j = \lambda_j v_j$$. Visible text: Let be an orthonormal basis consisting of eigenvectors of matrix . For each we have . Since it's an orthonormal basis, we have the following sequence of calculations: Component: MathContainer Children: ```math (A^H v_i)^H v_j = v_i^H (Av_j) = v_i^H (\lambda_j v_j) ``` ```math = \lambda_j v_i^H v_j = \lambda_j \delta_{ij} ``` This means $$A^H v_i = \overline{\lambda_i} v_i$$, so that: Visible text: This means , so that: Component: MathContainer Children: ```math AA^H v_i = A(\overline{\lambda_i} v_i) = \overline{\lambda_i} \lambda_i v_i ``` ```math = \lambda_i \overline{\lambda_i} v_i = \lambda_i (A^H v_i) = A^H (\lambda_i v_i) = A^H A v_i ``` Since this holds for all basis vectors, then $$AA^H = A^H A$$, which means $$A$$ is normal. Visible text: Since this holds for all basis vectors, then , which means is normal. ## Proof from Normality to Diagonalization Now we prove the opposite direction using mathematical induction. Imagine building a multi-story house, we start from the ground floor and prove that each new floor can be built using results from the previous floor. ### Mathematical Induction Structure 1. **Base Step** For $$n = 0$$ (empty matrix), the statement is clearly true. 2. **Induction Hypothesis** Assume the statement is true for all normal matrices of size $$(n-1) \times (n-1)$$. 3. **Induction Step** Let $$A \in \mathbb{C}^{n \times n}$$ be normal. Based on the fundamental theorem of algebra, there exists an eigenvalue $$\lambda_1 \in \mathbb{C}$$ of $$A$$. Let $$v_1 \in \mathbb{C}^n$$ be a corresponding eigenvector with $$v_1^H v_1 = 1$$. We have $$Av_1 = \lambda_1 v_1$$ and from the properties of normal matrices, $$A^H v_1 = \overline{\lambda_1} v_1$$. Visible text: 1. **Base Step** For (empty matrix), the statement is clearly true. 2. **Induction Hypothesis** Assume the statement is true for all normal matrices of size . 3. **Induction Step** Let be normal. Based on the fundamental theorem of algebra, there exists an eigenvalue of . Let be a corresponding eigenvector with . We have and from the properties of normal matrices, . ### Formation of Invariant Subspace Complete $$v_1$$ to an orthonormal basis of $$\mathbb{C}^n$$ with vectors $$v_2, \ldots, v_n$$. Define: Visible text: Complete to an orthonormal basis of with vectors . Define: ```math W = \text{span}(v_2, \ldots, v_n) ``` Here the span of vectors $$v_2, \ldots, v_n$$ is the set of all linear combinations of those vectors. In other words, $$W$$ contains all vectors that can be written as $$a_2 v_2 + a_3 v_3 + \cdots + a_n v_n$$ with $$a_2, a_3, \ldots, a_n \in \mathbb{C}$$. Visible text: Here the span of vectors is the set of all linear combinations of those vectors. In other words, contains all vectors that can be written as with . For every $$w \in W$$, we have: Visible text: For every , we have: Component: MathContainer Children: ```math (Aw)^H v_1 = w^H (A^H v_1) = w^H (\overline{\lambda_1} v_1) ``` ```math = \overline{\lambda_1} w^H v_1 = 0 ``` So $$Aw \in W$$, which means: Visible text: So , which means: ```math A(W) = \{Aw \mid w \in W\} \subset W ``` In other words, $$W$$ is a subspace that is invariant (unchanging) under transformation $$A$$. This is like a separate pond where fish swimming in that pond never leave the pond. Visible text: In other words, is a subspace that is invariant (unchanging) under transformation . This is like a separate pond where fish swimming in that pond never leave the pond. ### Unitary Transformation and Preservation of Normality Let $$T \in \mathbb{C}^{n \times n}$$ be a unitary matrix with columns $$v_1, \ldots, v_n$$. Then $$T \cdot T^H = T \cdot T^{-1} = I$$. Visible text: Let be a unitary matrix with columns . Then . This transformation gives an elegant block form: Component: MathContainer Children: ```math T^H \cdot A \cdot T = T^{-1} \cdot A \cdot T ``` ```math = \begin{pmatrix} \lambda_1 & 0^T \\ 0 & A' \end{pmatrix} ``` with: ```math A' \in \mathbb{C}^{(n-1) \times (n-1)} ``` Since unitary transformations preserve normality like a mirror that preserves the shape of objects, we can show that: Component: MathContainer Children: ```math (T^H \cdot A \cdot T) \cdot (T^H \cdot A \cdot T)^H ``` ```math = T^H \cdot A \cdot T \cdot T^H \cdot A^H \cdot T ``` ```math = T^H \cdot A \cdot A^H \cdot T ``` ```math = T^H \cdot A^H \cdot A \cdot T ``` ```math = (T^H \cdot A \cdot T)^H \cdot (T^H \cdot A \cdot T) ``` Therefore the block diagonal matrix is also normal, which means is also normal. ## Construction of Complete Orthonormal Basis Now we arrive at the final stage like assembling a puzzle that is almost complete. Based on the induction hypothesis, there exists an orthonormal basis of eigenvectors for . Let be a unitary matrix with those columns, where: ```math S' \in \mathbb{C}^{(n-1) \times (n-1)} ``` so that: ```math S'^H \cdot A' \cdot S' = \begin{pmatrix} \lambda_2 & & \\ & \ddots & \\ & & \lambda_n \end{pmatrix} ``` with eigenvalues $$\lambda_2, \ldots, \lambda_n$$ of . Visible text: with eigenvalues of . Now we define: ```math S = T \cdot \begin{pmatrix} 1 & 0^T \\ 0 & S' \end{pmatrix} ``` Then $$S$$ is a unitary matrix with: Visible text: Then is a unitary matrix with: ```math S \in \mathbb{C}^{n \times n} ``` and: Component: MathContainer Children: ```math S^H \cdot A \cdot S = \begin{pmatrix} 1 & 0^T \\ 0 & S'^H \end{pmatrix} \cdot T^H \cdot A \cdot T ``` ```math \cdot \begin{pmatrix} 1 & 0^T \\ 0 & S' \end{pmatrix} ``` ```math = \begin{pmatrix} \lambda_1 & 0^T \\ 0 & S'^H A' S' \end{pmatrix} ``` ```math = \begin{pmatrix} \lambda_1 & & \\ & \ddots & \\ & & \lambda_n \end{pmatrix} ``` The columns of $$S$$ form an orthonormal basis of eigenvectors of matrix $$A$$. Visible text: The columns of form an orthonormal basis of eigenvectors of matrix . ## Real Matrix Case For real matrices, the situation is slightly different like the difference between drawing on a flat canvas (real) compared to drawing in three-dimensional space (complex). A real matrix: ```math A \in \mathbb{R}^{n \times n} ``` is called normal if: ```math A \cdot A^T = A^T \cdot A ``` Normal real matrices are a special case of normal complex matrices with real entries. Therefore, the same properties apply to both types of symmetric and orthogonal matrices. Symmetric matrices and orthogonal matrices are always normal. For a symmetric matrix $$A^T = A$$, it is clear that: Visible text: Symmetric matrices and orthogonal matrices are always normal. For a symmetric matrix , it is clear that: ```math A \cdot A^T = A \cdot A = A^T \cdot A ``` For an orthogonal matrix $$A^T = A^{-1}$$, we have: Visible text: For an orthogonal matrix , we have: Component: MathContainer Children: ```math A \cdot A^T = A \cdot A^{-1} = I ``` ```math = A^{-1} \cdot A = A^T \cdot A ``` However, not all normal real matrices have real eigenvalues. In the real spectral theorem, the existence of real eigenvalues is not guaranteed, so orthogonal diagonalization may not always be possible in real numbers.