The Frisch-Waugh-Lovel theorem is a well-known result in theoretical economics. The theorem goes back to the 1933 paper of Frisch and Waugh¹. In this paper, they consider regressing an outcome variable on several covariates and a linear trend. They are interested in cancelling out the time trend from both outcome and covariates. What they show is that the coefficient for the covariates on the original regression can also be obtained by first detrending the outcome variable, by regressing it on a linear time trend, next detrending every covariates and finally by regressing the detrended outcome variable on the detrended covariates.

The follow-up 1963 JASA paper by Lovell² extends the Frish-Waugh result by looking at a richer model taking into account seasonal adjustments.

The OLS estimator

Before stating and proving the FWL theorem, I’ll quickly review the OLS estimator. Consider a dataset \((Y,X)\) where \(Y\) is an \(n\)-dimensional vector of outcomes (for \(n\) observations) and \(X\) is an \(n\times k\) matrix containing the values of \(k\) covariates over the \(n\) observations. We write \(X_j\) for the \(j\)-th column of \(X\), giving the values of the \(j\)th covariate.

If we regress \(Y\) on \(X\), we are trynig to find the linear combination of the columns of \(X\) that produces a vector that is as close to \(Y\) as possible. To be precise, we are looking for a \((k\times 1)\) vector \(b\) such that \(X b\) (which is of dimension \(n \times 1\)) is as close as possible to \(Y\).

Let \(L_X= \left\lbrace X \beta| \beta \in \mathbb{R}^k \right\rbrace\) be the set of \(n\) dimensional vectors spanned by the columns of \(X\). Regressing \(Y\) on \(X\) implies finding the element in \(L_X\) whose distance from \(Y\) is minimal.

$$\min\limits_{\hat Y \in L_X} ||Y – Y’||^2 = \min\limits_{\hat b \in \mathbb{R}^k} \|Y – X\hat b\|$$

The set \(L_X\) is a linear subspace. The vector \(\hat Y\) that solves the minimization problem is called the projection of \(Y\) onto the subspace \(L_X\). We know that \(\hat Y = X \hat b\) will solve the minimization problem if and only if every vector \(Z \in L_X\) is orthogonal to the vector \((Y – \hat Y)\). In other words:

$$ \forall Z \in L_X: Z'(Y – \hat Y) = 0.$$

Taking \(Z = X_j\) as special cases, we have the \(k\) conditions:

$$ X’_j (Y – X \hat b) = 0, \qquad \forall j = 1,\ldots, k.$$

Stacking all row vectors \(X_j\) on top of each other gives the matrix \(X’\), so we have a system of equations:

$$ X'(Y – X\hat b) = 0 \leftrightarrow \hat b = (X’X)^{-1} X’ Y,$$

which gives the standard OLS estimator we all know (and love). The value of \(\hat Y\) is given by

$$\hat Y = X \hat b = \underbrace{X(X’X)^{-1}X’}_{P_X} Y.$$

The \(n \times n\) matrix \(P_X\) is called the projection matrix of the linear subspace \(L_X\). The matrix is symmetric:

$$ P’_X = (X(X’X)^{-1}X’)’ = X(X’X)^{-1} X’,$$

and idempotent:

$$P_X P_X = X(X’X)^{-1} X’ X(X’X)^{-1} X’ = X(X’X)^{-1}X’ = P_X.$$

The residuals from the regression are given by:

$$ Y – \hat Y = Y – P_X Y = \underbrace{I – P_X}_{M_X} Y,$$

The annihilator matrix \(M_X\) is also symmetric (as it is the difference of two symmetric matrices) and idempotent:

$$ M_X M_X = (I – P_X)(I – P_X) = I – P_X – P_X + P_X P_X = I – P_X$$

Statement of the FWL theorem

Lest us separate the covariates \(X\) into two parts: an \((n \times k_A)\) matrix \(X_A\) and an \((n \times k_B)\) matrix \(X_B\). We can write \(X = [ X_A \,\, X_B]\). Let us now regress \(Y\) on \(X = [X_A\,\,X_B]\). Let \(\hat b_A\) and \(\hat b_B\) be the OLS estimates for the covariates in \(X_A\) and \(X_B\). Note that \(\hat b = [\hat b_A \,\, \hat b_B]\), so \(X \hat b = X_A \hat b_A  + X_B \hat b_B\).

Consider the following procedure:

1. Regress \(Y\) on \(X_B\). This produces the projection \(P_{X_B} Y\) and the residuals \(\hat Y = M_{X_B} Y\) where \(P_{X_B} = X_B (X_B’ X_B)^{-1} X_B\) and \(M_{X_B} = I – P_{X_B}\).
2. Regress each covariate in \(X_A\) on \(X_B\). This produces the projections \(P_{X_B} X_A\) and the residuals \(\hat X_A =  M_{X_B} X_A\).
3. Regress \(\hat Y\) from step 1 on \(\hat X_A\) obtained in step 2. This gives OLS estimates \(\hat \beta_A\).

The Frisch-Waugh-Lovell theorem  states that \(\hat \beta_A = \hat b_A\).

Proof

Consider the regression of \(Y\) on \(X\). The orthogonality conditions give:

$$ X’ (Y – X \hat b) = 0.$$

Writing out \(X = [X_A \, \, X_B]\) this gives:

$$ \begin{bmatrix} X_A’ \\ X_B’\end{bmatrix}(Y – X_A \hat b_A – X_B \hat b_B) = 0.$$

Splitting up the first \(k_A\) rows and the last \(k_B\) rows gives:

$$\begin{cases} X_A'(Y – X_A \hat b_A – X_B \hat b_B) = 0\\ X_B'(Y – X_A \hat b_A – X_B \hat b_B) = 0 \end{cases}$$

We want the value of \(\hat b_A\). We will do this by using the second equation to solve for \(\hat b_B\) and substitute this back into the first equation. From the second equation we get:

$$ \hat b_B = (X_B’ X_B)^{-1} X_B’ Y – (X’_B X_B)^{-1} X_B’ X_A \hat b_A.$$

Then:

$$
\begin{align*}
&X_A’ X_A \hat b_A + X_A’ X_B \hat b_B = X_A’ Y,\\
\leftrightarrow & X_A’ X_A \hat b_A + X_A’ X_B(X_B’X_B)^{-1}X_B’ Y – X_A’ X_B(X_B’ X_B)^{-1} X_B’ X_A \hat b_A = X_A’ Y,\\
\leftrightarrow &X_A’ X_A \hat b_A – X_A’ P_{X_B} X_A \hat b_A = X_A’ Y – X_A’ P_{X_B} Y,\\
\leftrightarrow &X_A'(I – P_{X_B})X_A \hat b_A = X_A'(I – P_{X_B})Y,\\
\leftrightarrow &X_A’M_{X_B}’ M_{X_B} \hat b_A = X_A’ M_{X_B}’ M_{X_B} Y,\\
\leftrightarrow &\hat X_A’ \hat X_A \hat b_A = \hat X_A’ \hat Y,\\
\leftrightarrow &\hat b_A = (\hat X_A’ \hat X_A)^{-1}\hat X_A’ \hat Y.
\end{align*}
$$

So we see that \(\hat b_A\) indeed coincides with the OLS estimate of the regression of \(\hat Y\) on \(\hat X_A\).

A corollary

Note that:

$$
\begin{align*}
\hat b_A &= (\hat X_A’ \hat X_A)^{-1} \hat X_A’ Y,\\
&=(\hat X_A’ \hat X_A)^{-1} X_A’ M’_{X_B} M_{X_B} Y,\\
&=(\hat X_A’ \hat X_A)^{-1} X_A’ M_{X_B}’ Y,\\
&=(\hat X_A’ \hat X_A)^{-1} \hat X_A’ Y
\end{align*}
$$

So we see that \(\hat b_A\) is also equal to the OLS estimate of regressing \(Y\) on \(\hat X_A\).