The OLS Bias Problem and the Need for Instruments
Suppose we want to estimate the causal effect of a variable `X` on an outcome `Y`. The workhorse method, ordinary least squares (OLS), assumes that the error term `\varepsilon` is uncorrelated with the regressor `X`. In countless real-world settings—education on wages, prices on demand, institutions on growth—this assumption crumbles. The regressor is endogenous: it correlates with unobserved determinants of `Y`, creating a backdoor path that contaminates the OLS estimate.
Mathematical Derivation of OLS Bias
Consider the simple linear model (without intercept for clarity): ` Y = \beta X + \varepsilon ` The OLS estimator is `\hat{\beta}_{\text{OLS}} = \frac{\sum X_i Y_i}{\sum X_i^2} = \frac{\sum X_i (\beta X_i + \varepsilon_i)}{\sum X_i^2} = \beta + \frac{\sum X_i \varepsilon_i}{\sum X_i^2}`. As the sample size grows, the probability limit becomes: ` \operatorname*{plim}_{n \to \infty} \hat{\beta}_{\text{OLS}} = \beta + \frac{\operatorname{Cov}(X,\varepsilon)}{\operatorname{Var}(X)}. ` When `\operatorname{Cov}(X,\varepsilon) \neq 0`, OLS is inconsistent. The bias term `\frac{\operatorname{Cov}(X,\varepsilon)}{\operatorname{Var}(X)}` does not vanish with more data; it converges to a non-zero constant.
Classic sources of such endogeneity include omitted variable bias (an unobserved confounder `U` affects both `X` and `Y`), measurement error in `X`, and simultaneity (reverse causality). In each case, the regressor `X` is “corrupted” by a part that is correlated with the structural error.
Why We Need Instrumental Variables
Because the problem lies in the correlation between `X` and `\varepsilon`, we need an external source of variation that moves `X` but is orthogonal to the error term. This is the role of an instrumental variable `Z`. The idea is to isolate the “clean” (exogenous) part of `X` and use only that variation to identify `\beta`.
The search for instruments turns on deep institutional knowledge: a policy change, a natural experiment, or a biological lottery that affects the treatment without directly influencing the outcome. This is the heart of credible causal inference—without a valid instrument, we are stuck with biased and inconsistent OLS.
How IV Works, Characteristics of a Good Instrument, and the LATE Theorem
What IV Does: Decomposing `X`
Formally, consider a model with a single endogenous regressor `X` and an instrument `Z`. The structural equation is ` Y = \beta X + \varepsilon, \quad \text{with } \operatorname{Cov}(X,\varepsilon) \neq 0. ` The first-stage regression links `X` and `Z`: ` X = \pi Z + v, ` where `v` captures all other determinants of `X`. Substituting into the structural equation yields the reduced form for `Y`: ` Y = \beta (\pi Z + v) + \varepsilon = (\beta\pi) Z + (\beta v + \varepsilon). ` The IV estimator, in its simplest (Wald) form, is the ratio of the reduced-form coefficient to the first-stage coefficient: ` \hat{\beta}_{\text{IV}} = \frac{\widehat{\operatorname{Cov}}(Y,Z)}{\widehat{\operatorname{Cov}}(X,Z)}. ` Intuitively, the instrument `Z` creates experimental-like variation in `X`. The IV estimator uses only the part of `X` that is predicted by `Z`—the exogenous component—to trace out the causal response of `Y`.
Key insight: `X` can be thought of as having an endogenous component (correlated with `\varepsilon`) and an exogenous component driven by `Z`. IV extracts the exogenous variation, discarding the contaminated part.
Characteristics of a Good Instrument
For `Z` to be a valid instrument, it must satisfy three core conditions:
- Relevance: `\operatorname{Cov}(Z,X) \neq 0`. The instrument must have a meaningful effect on the endogenous regressor. Weak instruments (low correlation) lead to biased and imprecise IV estimates.
- Exogeneity (exclusion restriction): `\operatorname{Cov}(Z,\varepsilon) = 0`. The instrument must affect the outcome `Y` only through its effect on `X`. No direct path, no confounding with omitted variables.
- Monotonicity (for LATE interpretation, see below): The instrument should affect all individuals in the same direction (or at least not push some toward treatment and others away from it in a way that creates uninterpretable mixtures).
In practice, the exogeneity assumption is untestable and must be defended with economic reasoning, institutional detail, and falsification checks.
Why IV Estimates the Local Average Treatment Effect (LATE)
The simple constant-effects model `\beta` assumes that the causal effect is the same for everyone. When treatment effects are heterogeneous—some people benefit more from a policy than others—IV no longer identifies the average treatment effect (ATE) for the whole population. Instead, under standard conditions (Imbens & Angrist, 1994), IV identifies the Local Average Treatment Effect (LATE): the average causal effect for the subpopulation of compliers, i.e., individuals whose treatment status is changed by the instrument.
This insight follows from a potential outcomes framework. Suppose a binary instrument `Z \in \{0,1\}` and a binary treatment `D \in \{0,1\}`. Individuals can be classified into four types: always-takers, never-takers, compliers, and defiers. Under monotonicity (no defiers) and the exclusion restriction, the Wald estimator recovers: ` \hat{\beta}_{\text{IV}} = \frac{E[Y|Z=1] - E[Y|Z=0]}{E[D|Z=1] - E[D|Z=0]} = E[Y_i(1) - Y_i(0) \mid \text{Complier}]. ` Thus, IV speaks only to those whose behavior is moved by the instrument. This is not a weakness but a precise, policy-relevant parameter: it tells us the effect of the treatment on the marginal individuals induced to change by the instrument. External validity then hinges on how similar those compliers are to other populations of interest.
Just-Identified IV, Median Unbiasedness, Bootstrap Limitations, and Institutional Knowledge
The Just-Identified Case
When we have exactly one instrument for one endogenous regressor, the model is just-identified. In this setting, the IV estimator is simply the ratio of two covariance terms and cannot be tested for overidentifying restrictions. While it is consistent, its finite-sample distribution has some remarkable properties—and pitfalls.
The IV estimator is a ratio of two approximately normal random variables. As a result, it has no finite-sample moments: the mean and variance do not exist. This is not merely a theoretical curiosity; it implies that in small samples the estimator can exhibit extreme outliers, especially when the instrument is weak.
Median Unbiasedness of IV
Despite lacking a mean, the just-identified IV estimator is median-unbiased. That is, for any true `\beta`, ` P(\hat{\beta}_{\text{IV}} \leq \beta) = P(\hat{\beta}_{\text{IV}} \geq \beta) = \frac{1}{2}. ` In words, the estimator is as likely to overestimate as it is to underestimate the true parameter. This property holds exactly in the normal simultaneous equations model, giving a solid frequentist justification for IV point estimates even in finite samples—provided the instrument is valid and relevant.
Median unbiasedness, however, does not guarantee tight confidence intervals. With weak instruments, the distribution becomes bimodal and dispersed, making inference fragile.
Bootstrap Limitations for IV
The bootstrap, often a go-to tool for inference when asymptotic approximations are questionable, faces serious difficulties in IV settings, especially with weak instruments:
- Non-existence of moments: Because the IV estimator lacks finite-sample moments, the standard bootstrap (which resamples with replacement) can produce extremely erratic estimates and does not consistently estimate the sampling distribution’s tail behavior.
- Weak instruments: Under weak identification, the asymptotic distribution is non-standard and depends on nuisance parameters. The conventional bootstrap fails to replicate this distribution, leading to incorrect confidence intervals.
- Alternatives: Procedures like the Anderson-Rubin test, the conditional likelihood ratio test, or subsampling methods are preferred for reliable inference when identification is weak.
Therefore, applied researchers should be cautious: bootstrapping an IV regression does not automatically yield valid standard errors, especially when the first-stage `F`-statistic is low.
Concluding Remarks: Institutional Knowledge and the Art of Identification
The journey from an OLS regression to a credible IV estimate is paved with theoretical rigor and, crucially, institutional knowledge. Mathematics gives us the conditions for identification—relevance, exogeneity, monotonicity—but only deep contextual understanding can uncover a variable that plausibly satisfies them.
Whether it’s quarter of birth (Angrist & Krueger), distance to college (Card), or rainfall shocks (Miguel, Satyanath & Sergenti), the most compelling instruments are those anchored in a story. The LATE framework reminds us that we are identifying an effect for a specific, instrument-driven subgroup. This transparency forces us to be explicit about who is affected and why, strengthening the credibility of causal claims.
In the end, identification is not a purely statistical exercise. It is a blend of economic theory, historical knowledge, and careful empirical design. OLS bias is the default; IV, despite its finite-sample quirks and reliance on strong assumptions, remains one of the most powerful tools we have for uncovering causal relationships when nature or policy grants us a valid instrument.
Post a Comment