Product Moment Correlation

What Is Product Moment Correlation?

Product Moment Correlation, often called Pearson Correlation or simply denoted by $r$, is the most commonly used statistical measure to determine how strong and in what direction the linear relationship (straight-line pattern) is between two quantitative variables (numbers).

The value of $r$ tells us whether the two variables tend to move in the same direction (positive), opposite directions (negative), or if there is no linear relationship at all.

Correlation from Scatter Diagrams

The most intuitive way to understand the value of $r$ is by looking at how the data points are scattered on a diagram:

Strong Positive Correlation

$r$ approaching +1 means both variables tend to move in the same direction.

If your data points look like this (rising from bottom left to top right and tightly clustered), the $r$ value will be close to +1.

Weak Positive Correlation

$r$ being positive but close to 0 means both variables tend to move in the same direction, but not very strongly.

If the points still show an upward trend but are more scattered like this, the $r$ value is positive but smaller (closer to 0).

Strong Negative Correlation

$r$ approaching -1 means both variables tend to move in opposite directions.

If the points fall from top left to bottom right and are very tightly clustered, the $r$ value will be close to -1.

No Linear Correlation

$r$ approaching 0 means the two variables have no linear relationship.

When the points are scattered randomly without a clear linear pattern, the $r$ value will be close to 0.

How is r Calculated?

The Pearson correlation coefficient ($r$) essentially measures how synchronously two variables (X and Y) move relative to their own variations.

Imagine this:

1. Individual Variation:

   Each variable (X and Y) has its own variability. Some values fluctuate a lot (large variation), while others are stable (small variation). This is measured by $SS_{xx}$ for X and $SS_{yy}$ for Y (formulas below).

2. Joint Variation (Covariance):

   We also need to know how X and Y vary together. When X increases, does Y also tend to increase? Or decrease? This measure of joint variation is called covariance, calculated using $SS_{xy}$.

   • If $SS_{xy}$ is large and positive: X and Y often move in the same direction.
   • If $SS_{xy}$ is large and negative: X and Y often move in opposite directions.
   • If $SS_{xy}$ is close to zero: No clear pattern of joint movement.

3. Standardizing the Measure:

   The problem is that the value of $SS_{xy}$ (covariance) is heavily influenced by the units of the data. For example, the covariance between height (cm) and weight (kg) will have a different value if we measure height in meters and weight in grams, even if the relationship is the same.

   To overcome this, we need to standardize the covariance measure. This is done by dividing the covariance ($SS_{xy}$) by a measure of the individual variations (adjusted using square roots: $\sqrt{SS_{xx} SS_{yy}}$).

   $$r = \frac{\text{How much X and Y vary together}}{\text{Standardized measure of individual X and Y variations}} = \frac{SS_{xy}}{\sqrt{SS_{xx} SS_{yy}}}$$

The result of this division is $r$, the Pearson Correlation Coefficient. Because it's standardized, its value will always be between -1 and +1, regardless of the original data units. This allows us to compare the strength of linear relationships between different pairs of variables.

So, the value of $r$ is determined by comparing how strongly X and Y move together relative to how much they move individually.

Product Moment Correlation Formula

To calculate the value of $r$ precisely, we use formulas involving the Sum of Squares:

What are $SS_{xy}$, $SS_{xx}$, and $SS_{yy}$?

These measure how varied our data is:

1. $SS_{xx}$ (Sum of Squares for x): Measures how spread out the x data is from its mean.

   $$SS_{xx} = \sum (x - \bar{x})^2 = \sum x^2 - \frac{(\sum x)^2}{n}$$

2. $SS_{yy}$ (Sum of Squares for y): Measures how spread out the y data is from its mean.

   $$SS_{yy} = \sum (y - \bar{y})^2 = \sum y^2 - \frac{(\sum y)^2}{n}$$

3. $SS_{xy}$ (Sum of Products of deviations for x and y): Measures how x and y vary together.

   $$SS_{xy} = \sum (x - \bar{x})(y - \bar{y}) = \sum xy - \frac{(\sum x)(\sum y)}{n}$$

Key:

• $n$: Number of data pairs (x, y).
• $\sum x$, $\sum y$: Sum of all x and y values.
• $\sum x^2$, $\sum y^2$: Sum of the squares of each x and y value.
• $\sum xy$: Sum of the product of each x and y pair.
• $\bar{x}$, $\bar{y}$: Mean of x and y values.

By calculating these three SS values and plugging them into the formula for $r$, we get the Product Moment Correlation Coefficient.

Interpreting the Value of r

Once we have the value of $r$, we can interpret its strength and direction using the following general guidelines: