AIC/BIC Model Selection

ECON 3209 · Week 16, Lecture 3 · Kerala Agricultural University

Department of Development Economics, KAU

Autumn 2026

Learning Outcomes

By the end of this lecture, you should be able to:

  1. define AIC and BIC and explain the logic of penalised model fit
  2. compare multiple candidate models using information criteria
  3. distinguish between stepwise search and theory-driven specification choice
  4. compute AIC/BIC in Python for NPA prediction models
  5. combine fit metrics with economic reasoning to choose a final model

Why We Need Model Selection

  • A model that fits the sample extremely well may still perform poorly on new data.
  • Adding more regressors always helps in-sample fit, but may create overfitting.
  • Good model selection balances:
    • fit,
    • simplicity,
    • interpretability,
    • economic credibility.

AIC and BIC Formulas

\[AIC = -2\ln L + 2k\]

\[BIC = -2\ln L + k\ln n\]

  • \(L\) is the maximised likelihood,
  • \(k\) is the number of estimated parameters,
  • \(n\) is sample size.

Lower values are preferred.

How AIC and BIC Differ

  • Both criteria reward better fit through higher likelihood.
  • Both penalise large models.
  • BIC penalises complexity more heavily, especially when sample size is large.
  • So AIC often prefers richer models; BIC often prefers more parsimonious ones.

Comparing Candidate Models

Reading the Table

  • We prefer the model with the lowest AIC or BIC.
  • If AIC and BIC agree, the decision is easy.
  • If they disagree, ask:
    • Is the richer model economically meaningful?
    • Is the improvement in fit substantial?
    • Do we need forecasting accuracy or interpretability?

Theory-Driven versus Stepwise Thinking

  • Theory-driven modelling starts from economic reasoning and then checks fit.
  • Stepwise procedures add or remove variables mechanically based on a rule.
  • Stepwise methods can be useful for exploration, but they may ignore causal logic and produce unstable models.
  • In econometrics, theory should usually lead and criteria should support.

Brute-Force Search over Small Model Sets

Kerala Cooperative Bank Example

Suppose we want to predict branch-level NPA ratio for cooperative banks.

Candidate regressors could include loan growth, recovery rate, inspection score, digital adoption, branch size, and district dummies.

AIC/BIC help us compare specifications, but final reporting should still match economic understanding of default risk.

Out-of-Sample Check

Good Model Selection Habits

  • Use AIC/BIC to compare economically sensible models.
  • Report the variables and why they were chosen.
  • Check residual diagnostics and out-of-sample performance.
  • Do not confuse a low AIC with a causal guarantee.
  • For exams and practice, combine theory + diagnostics + information criteria.

Exercise

Use the starter code to compare three NPA models. Which one would you choose under AIC? Under BIC? If the answers differ, explain your final choice in one sentence.

Summary

  • ✅ AIC and BIC compare models by rewarding fit and penalising complexity.
  • ✅ BIC uses a stronger penalty, so it often chooses smaller models than AIC.
  • ✅ Model selection should support theory, not replace it.
  • ✅ A useful applied workflow combines information criteria, diagnostics, and out-of-sample checks.

Next Lecture

  • We begin panel data analysis with pooled OLS.
  • You will learn how repeated observations on Kerala districts change the regression setup.
  • The next week adds fixed effects and random effects to the story.