Positive Definite Matrices

Quadratic Form Definitions

For a real symmetric matrix $A\in\mathbb{R}^{n\times n}$ , consider the quadratic form

q(x)=x^TAx

The complex version uses a Hermitian matrix and $q(x)=x^HAx$ . The sign of this scalar classifies the matrix:

Class	Sign condition
Positive definite	$x^TAx>0$
Positive semidefinite	$x^TAx\ge0$
Negative definite	$x^TAx<0$
Negative semidefinite	$x^TAx\le0$
Indefinite	the quadratic form takes both positive and negative values

For the four definite or semidefinite classes, the inequality must hold for $every nonzero vector$ . Indefinite means that at least one vector gives a positive value and another gives a negative value.

$Symmetry$ is part of the usual definition. For a nonsymmetric real matrix,

x^TAx=x^T\left(\frac{A+A^T}{2}\right)x

This identity shows that the quadratic form depends only on the symmetric part of $A$ .

Spectral Criterion

The spectral theorem writes a real symmetric matrix as

A=Q\Lambda Q^T

In this decomposition, $Q$ is orthogonal and $\Lambda=\operatorname{diag}(\lambda_1,\ldots,\lambda_n)$ is real and diagonal. Substituting $y=Q^Tx$ gives

x^TAx=\sum_{i=1}^n\lambda_i y_i^2

The calculation gives:

$A$ is positive definite exactly when every $\lambda_i>0$ .
$A$ is positive semidefinite exactly when every $\lambda_i\ge0$ .
$A$ is indefinite exactly when it has both positive and negative eigenvalues.

Every diagonal entry of a positive definite matrix is positive because

a_{ii}=e_i^TAe_i>0

A matrix can have positive diagonal entries and still fail to be positive definite because the $cross terms$ also affect $x^TAx$ .

Ellipsoid Geometry

If $A$ is positive definite, then

E=\{x\in\mathbb{R}^n:x^TAx=1\}

The set $E$ is an ellipsoid centered at the origin. In eigenvector coordinates, its equation becomes

\sum_{i=1}^n\lambda_i y_i^2=1

This equation shows that the $principal axes$ of the ellipsoid follow the eigenvectors of $A$ . The $i$ th semiaxis has length $1/\sqrt{\lambda_i}$ . Small eigenvalues produce longer semiaxes, while large eigenvalues produce shorter semiaxes.

For $A=\operatorname{diag}(1,1/4,1/9)$ , the three eigenvalues give semiaxis lengths $1,2,3$ . Rotate the model to compare the three directions. The curves are cross-sections of the same ellipsoid, not different surfaces.

x^2+\frac{y^2}{4}+\frac{z^2}{9}=1

Eigenvalues Set the Semiaxis Lengths

Cross-sections of

x^2 + \frac{y^2}{4} + \frac{z^2}{9} = 1

. The semiaxes along x, y, z have lengths 1, 2, 3.

$x$: $[-1, 1]$
$y$: $[-2, 2]$
$z$: $[-3, 3]$
Lines: 9

Principal Minor Tests

Let $A_k$ denote the upper-left $k\times k$ block of a real symmetric or complex Hermitian matrix. Sylvester's criterion states

$A$ is positive definite if and only if every leading principal determinant is positive:

\det(A_k)>0,\quad k=1,\ldots,n

This leading-principal-minor test does not extend by replacing $>$ with $\ge$ . Positive semidefiniteness requires $every principal minor$ to be nonnegative. Checking the leading ones alone is insufficient.

Worked Two by Two Checks

Consider

A=\begin{pmatrix}5&1\\1&4\end{pmatrix}

Its leading principal minors are

\det(A_1)=5>0

Both $leading principal minors are positive$ , so $A$ is positive definite. Both eigenvalues are also positive:

\begin{aligned} &\lambda_1=\frac{9+\sqrt5}{2}\approx5.61803, \\ &\lambda_2=\frac{9-\sqrt5}{2}\approx3.38197 \end{aligned}

Now compare

B=\begin{pmatrix}1&5\\5&4\end{pmatrix}

Although both diagonal entries are positive,

\det(B)=4-25=-21<0

The negative determinant shows that the two eigenvalues of $B$ have opposite signs. Therefore, $B$ is indefinite.

Cholesky Characterization

A real symmetric matrix is positive definite $exactly when it has a Cholesky factorization$ :

A=LL^T

In this factorization, $L$ is lower triangular and has strictly positive diagonal entries. Then

\begin{aligned} &x^TAx=x^TLL^Tx=\|L^Tx\|_2^2>0 \\ &(x\ne0) \end{aligned}

Conversely, the Cholesky algorithm completes without a nonpositive pivot for every positive definite matrix. For a symmetric input in exact arithmetic, this completion tests positive definiteness. Once $A=LL^T$ is available, solve $Ly=b$ and then $L^Tx=y$ .

Products with Swapped Rows and Columns

For every real matrix $M\in\mathbb{R}^{m\times n}$ ,

x^TM^TMx=\|Mx\|_2^2\ge0

This equation shows that $M^TM$ is always positive semidefinite. The matrix $M^TM$ is positive definite exactly when

\begin{aligned} &\ker(M)=\{0\} \\ &\Longleftrightarrow \\ &\operatorname{rank}(M)=n \end{aligned}

Full column rank automatically implies $m\ge n$ . It does not need to be assumed before the semidefinite statement.

More generally, a Gram matrix $G_{ij}=\langle v_i,v_j\rangle$ is positive semidefinite and becomes positive definite exactly when the vectors $v_i$ are $linearly independent$ .

Unique Minimizer of a Quadratic Objective

Consider the quadratic objective

F(x)=\frac12x^TAx-b^Tx+c

Its gradient and Hessian are

\nabla F(x)=Ax-b

If $A$ is positive definite, $F$ is $strictly convex$ and has the unique minimizer

x_*=A^{-1}b

The numerical difficulty also depends on the condition number, defined as the ratio of the largest to the smallest eigenvalue:

\kappa_2(A)=\frac{\lambda_{\max}(A)}{\lambda_{\min}(A)}

A large condition number gives elongated contours. Numerical algorithms may then run slowly or become sensitive to rounding error.

Spectral Shifts and Inverses

For Hermitian $A$ , the eigenvalues of $A-tI$ are $\lambda_i(A)-t$ . Hence

A-tI\succ0\quad\Longleftrightarrow\quad t<\lambda_{\min}(A)

If $A\succ0$ , then $A^{-1}\succ0$ as well, with eigenvalues $1/\lambda_i(A)$ . In numerical algorithms, $solve the linear system directly$ because forming an explicit inverse requires more operations and can amplify rounding error.