Product Moment Correlation

Direction and Strength of a Linear Relationship

Product moment correlation, often called Pearson correlation and denoted by $r$, is a common statistical measure of the direction and strength of the linear relationship between two quantitative variables.

The value of $r$ tells us whether the two variables tend to move in the same direction (positive), opposite directions (negative), or if there is no linear relationship at all.

Correlation from Scatter Diagrams

A scatter diagram shows the direction and strength represented by $r$. Points rising from left to right suggest a positive correlation, points falling from left to right suggest a negative one, and a shapeless cloud suggests none. The correlation coefficient turns that impression into a single number.

Strong Positive Correlation

$r$ approaching $+1$ means both variables tend to move in the same direction.

If your data points look like this (rising from bottom left to top right and tightly clustered), the $r$ value will be close to $+1$.

Weak Positive Correlation

$r$ being positive but close to $0$ means both variables tend to move in the same direction, but not very strongly.

For these eight points, the calculation gives $r\approx0.279$. The upward linear tendency is positive but weak because the points remain widely scattered around it.

Strong Negative Correlation

$r$ approaching $-1$ means both variables tend to move in opposite directions.

If the points fall from top left to bottom right and are very tightly clustered, the $r$ value will be close to $-1$.

Linear Correlation Close to Zero

$r$ approaching $0$ means the two variables show little or no linear relationship.

For the plotted points, $r\approx-0.050$. The linear association is very weak, rather than exactly zero. A nonlinear pattern still needs checking even when Pearson correlation is near zero.

Computing the Correlation Coefficient Step by Step

The Pearson correlation coefficient ($r$) compares the paired deviations of two variables, $X$ and $Y$, with the variation in each variable.

The calculation has three parts:

Separate the spread of each variable from the tendency of their paired deviations. This keeps the units of the two variables from determining the magnitude of the correlation coefficient.

The Spread of Each Variable

$X$ and $Y$ can have different amounts of spread. $SS_{xx}$ is the sum of the squared deviations of the $X$ values from the mean of $X$, while $SS_{yy}$ measures the same quantity for $Y$. Both formulas appear below.

Paired Deviations

We also need to know how $X$ and $Y$ vary together. When $X$ increases, does $Y$ also tend to increase or decrease? The sum $SS_{xy}$ measures these paired cross-deviations. It is the numerator used to calculate covariance. Covariance additionally divides by $n$ or $n-1$, depending on the convention.

• If $SS_{xy}$ is positive, paired deviations tend to have the same sign.
• If $SS_{xy}$ is negative, paired deviations tend to have opposite signs.
• If $SS_{xy}$ is close to zero, there is little net linear co-variation.

Scaling by the Spread of Both Variables

The value of $SS_{xy}$ is heavily influenced by the units of the data. For example, the cross-deviation sum for height and weight changes when height is converted from centimeters to meters, even though the underlying relationship is unchanged.

To overcome this, we standardize the cross-deviation sum by dividing $SS_{xy}$ by the geometric combination of the two individual variation measures, $\sqrt{SS_{xx} SS_{yy}}$.

The result of this division is $r$, the Pearson correlation coefficient. Provided both variables have nonzero variation, standardization constrains its value to the interval from $-1$ to $+1$, regardless of the original units. Because every valid pair uses the same scale, linear associations from different pairs of variables can be compared directly.

$r$ compares the paired deviations of $X$ and $Y$ with the separate variation in each variable.

Product Moment Correlation Formula

To calculate the value of $r$ precisely, we use formulas involving the Sum of Squares:

What are $SS_{xy}$, $SS_{xx}$, and $SS_{yy}$?

The three quantities have different roles. $SS_{xx}$ and $SS_{yy}$ measure the spread of each variable, while $SS_{xy}$ measures paired variation:

1. $SS_{xx}$ (Sum of Squares for $x$): Measures how spread out the $x$ data is from its mean.

   $$\begin{aligned} SS_{xx} &= \sum (x - \bar{x})^2 \\ &= \sum x^2 - \frac{(\sum x)^2}{n} \end{aligned}$$

2. $SS_{yy}$ (Sum of Squares for $y$): Measures how spread out the $y$ data is from its mean.

   $$\begin{aligned} SS_{yy} &= \sum (y - \bar{y})^2 \\ &= \sum y^2 - \frac{(\sum y)^2}{n} \end{aligned}$$

3. $SS_{xy}$ (Sum of Products of deviations for $x$ and $y$): Measures how $x$ and $y$ vary together.

   $$\begin{aligned} SS_{xy} &= \sum (x - \bar{x})(y - \bar{y}) \\ &= \sum xy - \frac{(\sum x)(\sum y)}{n} \end{aligned}$$

Key:

• $n$: Number of data pairs $(x, y)$.
• $\sum x$, $\sum y$: Sum of all $x$ and $y$ values.
• $\sum x^2$, $\sum y^2$: Sum of the squares of each $x$ and $y$ value.
• $\sum xy$: Sum of the product of each $x$ and $y$ pair.
• $\bar{x}$, $\bar{y}$: Mean of $x$ and $y$ values.

By calculating these three $SS$ values and plugging them into the formula for $r$, we get the Product Moment Correlation Coefficient.

The formula requires $SS_{xx}>0$ and $SS_{yy}>0$. If either variable is constant, the denominator is zero and Pearson's $r$ is undefined.

Calculating Correlation for Eight Pairs

Use the weak-positive-correlation data. Calculate the product of each matching pair and the square of each value, then sum each column.

The final row gives the sum of each column. With $n=8$, the three deviation sums are:

Interpreting the Correlation Coefficient

Once we have the value of $r$, we can use the following rough descriptive guidelines. Researchers use these cutoffs as conventions, and other fields may choose different ones. Their practical meaning depends on the field, measurement quality, sample size, and context, so always inspect the scatter plot as well.