Command Palette

Search for a command to run...

Linear Methods of AI

Scalar Product

Definition of Scalar Product

A scalar product is a fundamental operation that allows us to compute the "multiplication" between two vectors with the result being a scalar number. Through every positive definite matrix, we can define a scalar product on vector spaces.

A scalar product on complex vector space Cn\mathbb{C}^n is a sesquilinear Hermitian form that is positive definite with three main properties.

Sesquilinear Property

For all vectors x,y,z∈Cnx, y, z \in \mathbb{C}^n and scalars λ,μ∈C\lambda, \mu \in \mathbb{C}, the scalar product satisfies:

⟨λx+μy,z⟩A=λ‾⟨x,z⟩A+μ‾⟨y,z⟩A\langle \lambda x + \mu y, z \rangle_A = \overline{\lambda} \langle x, z \rangle_A + \overline{\mu} \langle y, z \rangle_A
⟨x,λy+μz⟩A=λ⟨x,y⟩A+μ⟨x,z⟩A\langle x, \lambda y + \mu z \rangle_A = \lambda \langle x, y \rangle_A + \mu \langle x, z \rangle_A

This property is called sesquilinear because it is semilinear in the first argument and linear in the second argument.

Hermitian Property

For all vectors x,y∈Cnx, y \in \mathbb{C}^n, the following holds:

⟨y,x⟩A=⟨x,y⟩A‾\langle y, x \rangle_A = \overline{\langle x, y \rangle_A}

Positive Definite Property

For all vectors x∈Cnx \in \mathbb{C}^n, the following holds:

⟨x,x⟩A≥0\langle x, x \rangle_A \geq 0
⟨x,x⟩A=0⇔x=0\langle x, x \rangle_A = 0 \Leftrightarrow x = 0

Thus, a scalar product is a sesquilinear Hermitian form that is positive definite on Cn\mathbb{C}^n.

Scalar Product on Real Space

If A∈Rn×nA \in \mathbb{R}^{n \times n} is a positive definite matrix, then the mapping:

⟨⋅,⋅⟩A:Rn×Rn→R:(x,y)↦⟨x,y⟩A=xTAy\langle \cdot, \cdot \rangle_A : \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R} : (x, y) \mapsto \langle x, y \rangle_A = x^T A y

has the following properties:

  1. Bilinear: For all vectors x,y,z∈Rnx, y, z \in \mathbb{R}^n and scalars λ,μ∈R\lambda, \mu \in \mathbb{R}, the following holds:

    ⟨λx+μy,z⟩A=λ⟨x,z⟩A+μ⟨y,z⟩A\langle \lambda x + \mu y, z \rangle_A = \lambda \langle x, z \rangle_A + \mu \langle y, z \rangle_A
    ⟨x,λy+μz⟩A=λ⟨x,y⟩A+μ⟨x,z⟩A\langle x, \lambda y + \mu z \rangle_A = \lambda \langle x, y \rangle_A + \mu \langle x, z \rangle_A

    This property is called bilinear because it is linear in both arguments.

  2. Symmetric: For all vectors x,y∈Rnx, y \in \mathbb{R}^n, the following holds:

    ⟨y,x⟩A=⟨x,y⟩A\langle y, x \rangle_A = \langle x, y \rangle_A

  3. Positive Definite: For all vectors x∈Rnx \in \mathbb{R}^n, the following holds:

    ⟨x,x⟩A≥0\langle x, x \rangle_A \geq 0
    ⟨x,x⟩A=0⇔x=0\langle x, x \rangle_A = 0 \Leftrightarrow x = 0

    Thus, a scalar product is a symmetric bilinear form that is positive definite on Rn\mathbb{R}^n.

Scalar Product on Complex Space

Generalization to complex space involves the important role of complex conjugate. If A∈Cn×nA \in \mathbb{C}^{n \times n} is a positive definite matrix, then the mapping:

⟨⋅,⋅⟩A:Cn×Cn→C:(x,y)↦⟨x,y⟩A=xHAy\langle \cdot, \cdot \rangle_A : \mathbb{C}^n \times \mathbb{C}^n \to \mathbb{C} : (x, y) \mapsto \langle x, y \rangle_A = x^H A y

has the same properties as the general definition above, namely sesquilinear in the first argument, linear in the second argument, Hermitian with complex conjugate, and positive definite.

This shows that when we move to complex space, the complex conjugate plays an important role in maintaining a consistent scalar product structure.

Matrix Representation

An important result in scalar product theory is that every scalar product can be expressed in the form of an appropriate matrix.

Every scalar product s(x,y)s(x, y) on Rn\mathbb{R}^n as well as on Cn\mathbb{C}^n can be expressed in the form:

s(x,y)=xTAy (for real case)s(x, y) = x^T A y \text{ (for real case)}
s(x,y)=xHAy (for complex case)s(x, y) = x^H A y \text{ (for complex case)}

through an appropriate positive definite matrix A∈Rn×nA \in \mathbb{R}^{n \times n} or A∈Cn×nA \in \mathbb{C}^{n \times n}.

The elements of matrix AA can be computed as:

aij=s(ei,ej) for i,j=1,…,na_{ij} = s(e_i, e_j) \text{ for } i, j = 1, \ldots, n

where eie_i is the standard basis vector.

Example of Standard Scalar Product

The simplest example is the standard scalar product that uses the identity matrix II.

The standard scalar product is defined as:

⟨x,y⟩=xTy (for real case)\langle x, y \rangle = x^T y \text{ (for real case)}
⟨x,y⟩=xHy (for complex case)\langle x, y \rangle = x^H y \text{ (for complex case)}

This standard scalar product is obtained by using the identity matrix as its representation matrix, so that all required properties are satisfied naturally.

Through the relationship between scalar products and positive definite matrices, we can construct various types of scalar products that suit specific application needs.

Previous

Positive Definite Matrix

Next

Matrix Condition