Multilevel and Mixed Models with Stata and LLMs - A Short Course

After introducing the key concepts of within and between variance, we will begin with simple multilevel variance components models that can tell us how much of the variance in a measure can be allocated to different levels of observation. We will then move on to mixed models (random effects models with fixed covariates) that allow us to ask how factors at different levels can influence the outcome.

Next, we will investigate how using random coefficients and cross-level interactions can help us discover hidden structure in our data and help us investigate how individual-level processes work differently in different contexts. We will also briefly consider how these techniques can be applied to cases where we have repeated observations of individuals or other entities over time.

Although the course will focus primarily on the continuous outcome case, we will also cover how these models can easily be extended for use with categorical and limited dependent variables.

The seminar will focus on a hands-on understanding and draw from examples across the social and behavioral sciences. After completing the course, you will know:

1. 1. The technical and substantive difference between fixed and random effects and how these terms relate to complete, partial, and no pooling estimators.
   2. The meanings of random intercepts and random slopes are and when to use each one.
   3. How to use cross-level interactions to investigate effect heterogeneity.
   4. How to combine the strengths of random-effects and fixed-effects approaches into a single “between-within” model.
   5. How to estimate these models and interpret the results with the assistance of LLMs.

Although these techniques apply to both clustered and longitudinal data, in the interest of time we will focus almost exclusively on the clustered case. For courses focused on longitudinal data analysis, check out Longitudinal Data Analysis Using R or Longitudinal Data Analysis with Stata and LLMs.

The majority of what you will learn in this course can be applied in any software package. This seminar will use the most recent version of Stata for empirical examples and exercises. (Nearly all commands will work in Stata 14+ as well.)

For LLM support, the instructor will use the most recent paid version of ChatGPT. However, most modern LLMs (e.g., Claude, Gemini) will be useful for understanding, modifying, and interpreting mixed models.

Basic familiarity with Stata is highly desirable, but even novice Stata users should be able to follow the presentation and do the exercises.

R notes and syntax are available upon request.

If you’d like to familiarize yourself with Stata basics before the seminar begins, we recommend following along with a “getting started” video like the one here.

Seminar participants who are not yet ready to purchase Stata could take advantage of StataCorp’s 30-day software return policy.

This course is for anyone who wants to learn to apply multilevel models to observational data. You should have a basic foundation in linear regression.

Module 1: Preliminaries

• • Course goals and the “big picture” of multilevel/mixed modeling
  • Variance as the core idea: residual variance and variance “explained”
  • Why multilevel models: separating within vs. between variability
  • Common data structures: clusters (e.g., students-in-schools) vs. repeated measures (panels)
  • Quantifying clustering with VPC/ICC (including worked examples)
  • LLMs: giving the context of your data

Module 2: Models with one dimension

• • Pooling decisions: complete pooling vs. no pooling vs. partial pooling
  • Partial pooling and shrinkage intuition (Empirical Bayes/BLUP ideas)
  • Random effects vs. fixed effects: meaning in one dimension
  • Building and interpreting a basic random-intercept model in Stata (mixed)
  • A full working example (data description, distributions, country/sample-size issues)
  • Visualizing group effects with caterpillar plots
  • Basic model selection (BIC, with interpretation)
  • LLMs: interrogating sample sizes and shrinkage estimates

Module 3: Two dimensions

• • Moving to two dimensions: random intercepts and random slopes
  • When do you need variable slopes? Diagnostics and decision logic
  • Practical modeling details: scaling predictors, quadratics, and interpretation of coefficients
  • LLMs: model assumptions and interpretation; how (and whether) to “relax” model assumptions

Module 4: Multivariate models

• • Adding predictors at Level 1 vs. Level 2
  • Variance explained by level; reporting R²-style summaries
  • Cross-level interactions as “manifest” moderators of random slopes
  • RE vs. FE vs. correlated random effects (CRE)/between-within approaches
  • Hausman logic and pitfalls
  • LLMs: key prompts for assessing model accuracy and robustness

Module 5: Additional topics

• • Three-level models: equation, Stata syntax, and three-level VPC
  • Binary outcomes: mixed logistic regression (melogit), ICC/VPC issues, odds ratios, visualization, and CRE logit variant
  • Presenting results: conventional table sequence (null → L1 → L2 → cross-level), plus plotting interactions and suggested references

Module 6: Repeated measures data

• • Extension of core ideas to panel data using a worked example
  • LLMs: useful prompts for repeated measures data

“Stephen Vaisey is an incredible instructor. He builds conceptual frameworks effectively, taking us step by step through the early stages so we can best understand final model specification. He’s an outstanding teacher.”
  Starr Solomon, Kent State University

“I appreciated the instructor’s ability to communicate statistics in a way that both engaged the listener and drilled concepts with the use of several examples and repetition of key concepts.”
  Maria Hershey, Harvard TH Chan School of Public Health

“I liked that this course was taught in a non-technical, non-jargon way, and helped me understand the key fundamentals.”
  Meril Antony, Rutgers University

“I really loved this course; it’s been one of the most helpful statistical seminars I’ve ever taken (including in grad school)! I appreciated the use of real survey data for the lectures and exercises. By far, the most helpful thing about this course was the emphasis placed on the basic understanding of the variance components (as in a single-level regression model), where they came from, and how they could be generalized to a multilevel context. I also really liked the characterization of the fixed and random slopes and intercepts as one of three pooling variations. It really helped simplify things, and I finished this course feeling like I understood multilevel modeling much better.”
  Ally Williams, Beech Acres Parenting Center

“This course was excellent. I’ve read MLMM articles, I’ve read MLMM books, and I’ve taken an MLMM course from a different company. This course was more valuable than all of that because Professor Vaisey provides a strong and coherent conceptual framework for thinking about MLMMs, and for how to address multilevel research questions through an MLMM analysis. While the course covers the particular syntax to run to produce models, the emphasis is really on how to fruitfully think about these things so that one can obtain proper models addressing one’s question of interest. Professor Vaisey has it all at his fingertips and is really good at explaining and illustrating statistical concepts, both verbally and through demonstrations in code.”
  Nick Huntington, Brandeis University