# Nakafa Framework: LLM

URL: /en/subject/university/bachelor/ai-ds/linear-methods/linear-model
Source: https://raw.githubusercontent.com/nakafaai/nakafa.com/refs/heads/main/packages/contents/subject/university/bachelor/ai-ds/linear-methods/linear-model/en.mdx

Output docs content for large language models.

---

export const metadata = {
    title: "Linear Model",
    description: "Learn linear models: fundamental y=h(t)·x relationships, polynomial/trigonometric forms, data-to-model conversion, and linearization techniques for AI.",
    authors: [{ name: "Nabil Akbarazzima Fatih" }],
    date: "07/14/2025",
    subject: "Linear Methods of AI",
};

## Understanding Linear Models

Imagine we are trying to understand relationships between various things in the real world. 
Linear models are a very elegant way to describe these relationships in mathematical form.

The basic form is very simple:

<BlockMath math="y = h(t) \cdot x" />

Why is it called "linear"? Because if we look at the relationship between <InlineMath math="y" /> and <InlineMath math="x" />, 
the relationship forms a straight line. Here <InlineMath math="h(t)" /> acts as a "connector" 
with dimensions <InlineMath math="\mathbb{R}^{d \times n}" />.

Let's get to know the three main players in this model:
- <InlineMath math="y \in \mathbb{R}^d" /> is the **result we observe** (model response)
- <InlineMath math="x \in \mathbb{R}^n" /> is the **value we want to find** (model parameters)  
- <InlineMath math="t \in \mathbb{R}^k" /> is the **input we provide** (independent variables)

What's interesting is that although we call it "linear", the relationship with input <InlineMath math="t" /> 
can actually be complex or curved. Only the relationship with parameter <InlineMath math="x" /> is linear.

## From Data to Model

Now, how do we use this model in real life? 
The process is actually like playing detective with data.

First, we conduct a series of experiments or measurements:

1. We choose various values for <InlineMath math="t" />
2. For each of these values, we measure and obtain <InlineMath math="y" />
3. Our goal is to find <InlineMath math="x" /> that can explain all this data

Suppose we perform <InlineMath math="M" /> measurements. 
For each measurement <InlineMath math="i" />, we have:

<BlockMath math="y_i \approx h(t_i) \cdot x, \quad i = 1, \ldots, M" />

Why use the "approximately equal" sign instead of "equals"? 
Because in the real world, no measurement is perfect. 
There's always noise, instrument errors, or other random factors that affect the results.

If we count the total of all data we collect, the amount is <InlineMath math="m = M \cdot d" />. 
Usually this number is much larger than the number of parameters we want to find (<InlineMath math="n" />), 
so <InlineMath math="m \gg n" />.

Our challenge now is how to find the best value of <InlineMath math="x" />, 
so that the equation <InlineMath math="y_i = h(t_i) \cdot x" /> is satisfied as accurately as possible.

When we arrange all this data, a system of equations is formed that looks like this:

<MathContainer>
<BlockMath math="b = \begin{pmatrix} y_1 \\ \vdots \\ y_M \end{pmatrix} = \begin{pmatrix} h(t_1) \\ \vdots \\ h(t_M) \end{pmatrix} \cdot \begin{pmatrix} x_1 \\ \vdots \\ x_n \end{pmatrix} = A(t) \cdot x" />
</MathContainer>

This produces a system with matrix <InlineMath math="A(t)" /> of size <InlineMath math="m \times n" /> 
and vector <InlineMath math="b" /> of size <InlineMath math="m" />.

## Various Forms of Linear Models

Linear models turn out to be very flexible and can take various forms. 
Let's look at some examples that appear most frequently:

### Simple Straight Line

The most basic form is a straight line:

<BlockMath math="y = a + b \cdot t = (1 \quad t) \cdot \begin{pmatrix} a \\ b \end{pmatrix} = h(t) \cdot x" />

Here we look for two parameters: <InlineMath math="a" /> (intercept) and <InlineMath math="b" /> (slope). 
This model is suitable when data forms a straight line pattern or nearly straight.

### Polynomial Curves

If data forms curved patterns, we can use polynomials:

<BlockMath math="y = a_0 + a_1 \cdot t + \ldots + a_n \cdot t^n = (1 \quad t \quad \ldots \quad t^n) \cdot \begin{pmatrix} a_0 \\ a_1 \\ \vdots \\ a_n \end{pmatrix} = h(t) \cdot x" />

Although <InlineMath math="t^n" /> looks nonlinear, remember that what we mean by "linear" 
is the relationship with parameters <InlineMath math="a_0, a_1, \ldots, a_n" />.

### Repeating Patterns with Trigonometry

For data that has repeating or cyclic patterns, we can use sine and cosine functions:

<BlockMath math="y = a_0 + \sum_{k=1}^{n} a_k \cos(k \cdot t) + \sum_{k=1}^{n} b_k \sin(k \cdot t)" />

This model is very useful for analyzing data that has seasonal or periodic patterns.

### Multiple Inputs

If the output depends on several inputs simultaneously, we can combine them. 
For example with two inputs <InlineMath math="t" /> and <InlineMath math="s" />:

<MathContainer>
<BlockMath math="y = a + b \cdot t + c \cdot s + d \cdot t \cdot s = (1 \quad t \quad s \quad t \cdot s) \cdot \begin{pmatrix} a \\ b \\ c \\ d \end{pmatrix} = h(t,s) \cdot x" />
</MathContainer>

The term <InlineMath math="t \cdot s" /> captures the interaction between the two inputs.

### Multiple Outputs

Sometimes we want to predict several things simultaneously from the same input:

<MathContainer>
<BlockMath math="y = a + b \cdot t" />
<BlockMath math="z = c + d \cdot t" />
</MathContainer>

This is like having two linear models running simultaneously.

## Real Example from Physics

The general gas equation in physics is:

<BlockMath math="p = n \cdot R \cdot \frac{T}{V}" />

Here:
- <InlineMath math="p" /> is the pressure we measure
- <InlineMath math="T" /> is the temperature we set
- <InlineMath math="V" /> is the volume we set  
- <InlineMath math="n" /> is the number of gas molecules (which we want to determine)
- <InlineMath math="R" /> is a known constant

If we consider <InlineMath math="T" /> and <InlineMath math="V" /> as inputs we can control, 
and <InlineMath math="p" /> as output we measure, then pressure depends linearly on <InlineMath math="n" />. 
This allows us to use linear models to determine the number of gas molecules.

## When Models Are Initially Nonlinear

Not all real-world problems are directly linear in form. 
Sometimes we encounter models where parameters appear in quadratic form, multiplication between parameters, 
or even in exponential functions:

<BlockMath math="y = h(t, x)" />

But don't despair! In many cases, we can "linearize" such models. 
The method is by using tangent line approximation around a certain point <InlineMath math="x_0" />:

<BlockMath math="y \approx h(x_0, t) + \frac{\partial h}{\partial x}(x_0, t) \cdot (x - x_0)" />

With this trick, we replace a complex curve with a straight line that approximates it. 
The result is an equation that is linear with respect to <InlineMath math="x" />, so it can be solved 
with standard linear algebra methods like least squares.

However, this linearization method only works if the nonlinear model is not too "curved" 
around point <InlineMath math="x_0" />. Highly nonlinear models require numerical optimization techniques.