Statistical Properties of OLS

ECON 3209 · Week 8, Lecture 2 · Kerala Agricultural University

Department of Development Economics, KAU

Autumn 2026

Learning Outcomes

  1. Explain the meaning of unbiasedness, consistency, and efficiency.
  2. Describe the sampling distribution of OLS estimators.
  3. Use simulation to see repeated-sample behaviour of \(\hat{\beta}_1\).
  4. Understand how sample size affects estimator precision.
  5. Connect OLS properties to later statistical inference.

Why estimator properties matter

  • A single regression output comes from one sample.
  • Econometric theory asks what would happen across many possible samples.
  • Good estimators behave well on average and improve with more data.
  • These properties justify confidence intervals and hypothesis tests.
  • OLS is attractive because of its strong theoretical foundations.

Unbiasedness

\[E(\hat{\beta}_1) = \beta_1\]

  • An estimator is unbiased if its average value over repeated samples equals the true parameter.
  • It does not mean every individual sample estimate is exact.
  • Some estimates will be above and some below the truth.
  • Under zero conditional mean, OLS is unbiased.

Consistency

  • An estimator is consistent if it converges to the true parameter as sample size grows.
  • Symbolically, \[\hat{\beta}_1 \xrightarrow{p} \beta_1\]
  • Large samples reduce random sampling noise.
  • Consistency is crucial in empirical work because no finite sample is perfect.
  • Bias from omitted variables may destroy consistency.

Efficiency

  • Among unbiased estimators, a more efficient estimator has smaller variance.
  • Smaller variance means estimates cluster more tightly around the truth.
  • Under the CLR assumptions, OLS is efficient within the class of linear unbiased estimators.
  • Efficiency improves the precision of inference.
  • This is the “best” part of BLUE.

Sampling distribution of OLS

  • Because samples vary, \(\hat{\beta}_1\) is a random variable.
  • Under normal errors, its sampling distribution is normal in small samples.
  • In large samples, it is approximately normal by asymptotic theory.
  • This supports the use of \(t\) tests and confidence intervals.
  • Standard errors measure the spread of this sampling distribution.

Simulating repeated samples

Seeing sampling variation

Sample size and consistency

What standard errors represent

  • The standard error estimates the standard deviation of the estimator.
  • A smaller standard error means more precise estimation.
  • Standard errors fall when sample size rises, holding everything else constant.
  • Noisy data and weak regressor variation raise standard errors.
  • Precision and bias are different concepts.

Empirical mean and spread of simulated slopes

Sum of squares decomposition

\[TSS = \sum_{i=1}^{n}(Y_i - \bar{Y})^2\]

\[ESS = \sum_{i=1}^{n}(\hat{Y}_i - \bar{Y})^2\]

\[RSS = \sum_{i=1}^{n}\hat{u}_i^2\]

\[TSS = ESS + RSS\]

  • TSS measures total variation in the dependent variable.
  • ESS is the variation explained by the fitted regression line.
  • RSS is the unexplained variation left in the residuals.

R-squared

\[R^2 = \frac{ESS}{TSS} = 1 - \frac{RSS}{TSS}, \qquad 0 \leq R^2 \leq 1\]

  • \(R^2\) tells us the share of variation in \(Y\) explained by the model.
  • A higher \(R^2\) means better fit, but not necessarily better causal identification.
  • A key limitation is that adding regressors never reduces \(R^2\), even if the extra variables add little real value.

Adjusted R-squared

\[\bar{R}^2 = 1 - \frac{(1-R^2)(n-1)}{n-k-1}\]

  • Here \(n\) is the sample size and \(k\) is the number of explanatory variables.
  • Adjusted \(R^2\) penalises unnecessary regressors by accounting for lost degrees of freedom.
  • Unlike ordinary \(R^2\), adjusted \(R^2\) can fall when a new variable adds almost no explanatory power.

Computing \(R^2\) and adjusted \(R^2\) manually in Python

When OLS properties break down

  • If \(E(u \mid X) \neq 0\), OLS is biased and inconsistent.
  • If error variance changes, usual inference becomes unreliable.
  • If the sample is not representative, estimates may not generalise well.
  • If regressors are measured with error, slopes may be attenuated.
  • Theory helps us diagnose what can go wrong.

Omitted variable bias in a quick simulation

Exercise

Simulate 200 samples from a simple linear model with a true slope of 0.4. Store the estimated slopes, compute their average, and explain whether the result supports unbiasedness.

Summary

  • ✅ Unbiasedness means the estimator is correct on average across repeated samples.
  • ✅ Consistency means the estimator approaches the true parameter as sample size grows.
  • ✅ Efficiency means lower variance among unbiased estimators.
  • ✅ OLS estimators have a sampling distribution, which underlies inference.
  • ✅ Larger samples usually improve precision.
  • ✅ Violations such as omitted variables can destroy desirable OLS properties.

Next Lecture

  • We will implement OLS in Python from scratch.
  • First with NumPy formulas, then with statsmodels.formula.api for a full regression workflow.