Hi, this is the Neutropic team!
Picture this: you have a dataset where every participant was measured several times, and your supervisor says, "That needs a hierarchical model. Try it Bayesian, in PyMC." You have run a repeated-measures ANOVA before, but hierarchical models are new, and Bayesian ones even newer. Open any example notebook and you are hit with priors, chains and divergences all at once. 🥲
The model itself is surprisingly short. The hard part comes after it. If you read numbers without checking that the chains mixed, why divergences appeared and whether the model can reproduce your data, you get a confident wrong answer.
In this post we follow Jordan, a psychology PhD student, setting up a first PyMC hierarchical model for repeated measures. Jordan is fictional, but every screen below comes from a real chat in Neutropic.
Jordan asked one short question
Jordan wanted the big picture before uploading any data, so the question was simply:
How should I set up and check a Bayesian hierarchical model in PyMC for repeated measures?
It is short, but it has what matters: the tool (PyMC), the data structure (repeated measures) and two verbs. "Set up" alone would have produced model code. Adding "and check" is what pulled the diagnostics into the answer.

It read a PyMC method guide before answering
As soon as Jordan sent it, steps appeared. Finding a method guide located a guide for hierarchical models and their diagnostics, and Reading a method guide opened the PyMC guide twice. There was no code-running step in this chat, so nothing was sampled.
Method guides are analysis procedures Neutropic has reviewed in advance. Instead of answering from memory, the model reads a vetted procedure first and follows it.
The model nests observations within participants
The answer opened with the idea: observations nested within participants, a population-level effect plus participant-specific deviations, and, if there is a within-person predictor, letting its effect vary by participant when the data can support it.
The example was a basic Gaussian model on long-format data, one row per measurement. The 47-line code gave each participant their own intercept and an optional slope for a within-person predictor, both written in a non-centred form, and drew prior predictive, posterior and posterior predictive samples.

Right under the code came a warning: the prior scales are illustrative only. Choose them from the units of your outcome and predictor, and look at prior predictive outcomes before fitting. The answer also noted that if the predictor is between-person rather than within-person, you should separate within-person and between-person effects.
The checks came as a list
The part Jordan liked most was the checklist, with three lines of code to start it.
- Before sampling: check that the priors imply plausible outcomes, that the design identifies the effects you want, and that the model starts with a finite log probability.
- Sampling: inspect rank R-hat, bulk and tail ESS, MCSE for the contrast you actually care about, divergences, energy/BFMI, and the traces.
- Posterior predictive checks: compare replicated and observed data, and also look at residuals by participant and over time.
- Robustness and prediction: check sensitivity to plausible priors, simulate data with known parameters to test recovery, and validate with splits that match your prediction target.

Jordan nearly took "more draws" as the fix
When a first Bayesian model shows divergences or low ESS, the instinct is to sample longer. Jordan was about to do just that. The answer warns against it: investigate divergences and poor mixing rather than treating more draws as an automatic fix, and consider reparameterising, stronger justified priors, or simplifying random effects the data cannot support.
Two more cautions stood out. A pooled posterior predictive plot can hide misfit at the participant level or over time. And holding out single observations answers a different question from holding out whole participants.

One more honest note: this chat did not run the model. The answer presents itself as a template for a continuous outcome with conditionally independent residuals, and the numbers in the code are settings, not results. Jordan still needs to install PyMC and run it locally.
It suggested what to do next
At the bottom were two suggestions:
- Extend the model to correlated participant-specific intercepts and slopes, including when that covariance is identifiable and how to diagnose it
- Compare the Gaussian model with suitable likelihoods for binary, ordinal and count outcomes, and explain what changes in the predictors and checks
Click one and the conversation continues in the same chat.

A vague "use PyMC" became code with a checklist
| At the start | Now | |
|---|---|---|
| Model | Only the phrase "hierarchical model" | Participant intercepts and optional slopes, non-centred |
| Code | Digging through example notebooks | 47 lines of PyMC code plus 3 lines of checks |
| Checks | Knew R-hat, roughly | Before sampling, sampling, PPC, robustness |
| Traps | No idea what to watch | Mixing, pooled plots, validation splits, priors |
| Next | Stuck | Two suggestions to click |
To be clear, this is not a sampled model. The code has to run on Jordan's own machine, and the prior scales need adjusting to the real data's units. If residuals stay correlated over time, the answer says to model that dependence rather than assume the participant effect removes it. Still, starting with a checklist in hand is very different from starting from a blank notebook.
This walkthrough used Pro. Lite (the free plan) gives a short model structure and the key diagnostics; Expert adds prior sensitivity and model comparison.
If repeated measures leave you stuck, one question gets you the model and the checks to run on it. 😎
This week, ask the way Jordan did: write "set up and check". Start with Neutropic for free