Multilevel and Mixed Models Using R and LLMs - A Short Course

After introducing the key concepts of within and between variance, we will begin with simple multilevel variance components models that can tell us how much of the variation in a measure can be attributed to different levels of observation. We will then move on to mixed models (random effects models with fixed covariates) that allow us to ask how factors at different levels can affect an outcome.

Next, we will investigate how using random coefficients and cross-level interactions can help us discover hidden structure in our data and investigate how individual-level processes work differently in different contexts. We will also briefly consider how these techniques can be applied to cases where we have repeated observations of individuals or other entities over time.

Although the course will focus primarily on the continuous outcome case, we will also briefly cover how these models can easily be extended for use with categorical and limited dependent variables.

The seminar will focus on hands-on understanding and draw from examples across the social and behavioral sciences. At the conclusion of the course, you will:

1. 1. Know the technical and substantive difference between fixed and random effects.
   2. Understand random intercepts and random coefficients and when to use each one.
   3. Know how to combine the strengths of random-effects and fixed effects approaches into a single model.
   4. Know how to estimate these models and interpret the results with the assistance of LLMs.

Although these techniques apply to both nested and longitudinal data, in the interest of time we will focus exclusively on the nested data case. For a course focused on longitudinal data analysis, check out Longitudinal Data Analysis Using R and LLMs.

The vast majority of what you will learn in this course can be applied in any software package. However, this seminar will mostly use R for empirical examples and exercises. To replicate the instructor’s workflow in the course, you should have R and RStudio already installed on your computer when the course begins.

For LLM support, the instructor will use the most recent paid version of Claude. However, most modern LLMs (e.g., ChatGPT, Gemini) will be useful for understanding, modifying, and interpreting mixed models.

Basic familiarity with R is highly desirable. If you are new to R, check out Professor Vaisey’s one-hour Introduction to R video to get up to speed. You may also want to consider taking a short introductory seminar on R such as Introduction to R for Data Analysis, R for SPSS Users, R for SAS Users, or R for Stata Users.

If you’d like to take this course but are concerned that you don’t know enough R, there are excellent online resources for learning the basics. Here are our recommendations.

Stata notes and syntax are available upon request.

This course is for anyone who wants to learn how to apply multilevel models to observational data. You should have a basic foundation in linear regression.

Module 1. Foundations of Multilevel Thinking

• What are multilevel models?
• Hierarchical linear model motivation and notation
• Within and between variance
• LLMs: giving the context of your data
• Variance components models and plots
• The first distinction between “random” and “fixed” effects
• “Shrinkage” and empirical Bayes estimates of intercepts
• LLMs: interrogating sample sizes and shrinkage estimates

Module 2. Building and Interpreting Mixed Models

• What are mixed models a mix of?
• Three types of R-squared
• Random coefficients/slopes
• LLMs: model assumptions and interpretation; how (and whether) to “relax” model assumptions
• Model selection with BIC and AIC
• Basic diagnostics

Module 3. Random vs. Fixed Effects and Cross-Level Modeling

• The second distinction between “random” and “fixed” effects
• When and why (not) to use random effects models
• Testing the random effects assumption
• The “between-within” method to combine the best of RE/FE
• Centering and cross-level interactions
• RE vs. FE vs. correlated random effects (CRE)/between-within approaches
• Hausman logic and pitfalls
• LLMs: key prompts for assessing model accuracy and robustness

Module 4. Extensions, Repeated Measures, and Communication

• A brief comparison of clustered and panel data
• LLMs: useful prompts for repeated measures data
• Multilevel logistic regression and other limited dependent variables
• Presenting results

“The course was excellent. Dr. Vaisey was able to break down the content into simple terms and make sure that the focus extended beyond running syntax in R. The course developed a conceptual understanding of multilevel models. The humor was also much appreciated.”
  Linda Zientek, Sam Houston State University

“I enjoyed how the course started with bringing about a deeper understanding of variance conceptually before moving on to the technicalities. Steve Vaisey’s dedication, pedagogical style and skills, and his deep knowledge of the topics covered during the course were great.”
  Henrik Gyllstad, Lund University

“I enjoyed Dr. Vaisey’s personable teaching style. He started with simple concepts, emphasizing the principles, then built on them. He was also very responsive to questions and made the course highly interactive.”
  Tom Tolbert, Icahn School of Medicine at Mount Sinai

“Terrific course! Dr. Vaisey was able to explain things in a simple, straightforward fashion, and I appreciated the practical examples.”
  Lauren Ilano, California State University

“Dr. Vaisey is a superb educator. He explained concepts in a clear and concise manner while adding bits of humor. I liked that he was quick to answer all questions, and was thorough in explaining the multilevel models and coding.”
  Alice Daugherty, The University of Alabama