How to size a class-randomised experiment with Neutropic: from one question to the number of classes

When whole classes are assigned to conditions, "how many students?" becomes "how many classes?". Follow Priya, an education master's student, from a one-line question to the design effect, a worked example and the caveats that come with it, all in real Neutropic screens.

Hi, this is the Neutropic team!

If you plan a classroom intervention study, you will almost certainly hit this wall. The school will not let you randomise students one by one. Timetables and rooms are fixed, so the most you can do is "this class gets the new lesson, that class keeps the old one". Then your committee asks: "So what's your sample size?" 🥲

The common mistake is to treat students as independent, run an ordinary two-group power analysis and use that number. Students in the same class share a teacher, a room and a routine, so their outcomes are alike. That means 200 students in 8 classes carry far less information than 200 students randomised individually.

In this post we follow Priya, a master's student in education, sizing a class-randomised experiment for the first time. Priya is fictional, but every screen below comes from a real chat in Neutropic.

One fact, no numbers

Priya had not settled on an effect size or class size yet, so the question was just the situation:

I'm comparing two groups in an experiment, but whole classes are assigned to each condition. How should I calculate the sample size?

It is short, but it carries the one fact that changes the whole calculation: whole classes are assigned. Leave that out and you get a standard two-group answer. And you do not need every input up front. When numbers are missing, Neutropic works through an example with its assumptions stated.

The steps under the request: method guides found and read, a power analysis and code run, then a literature search.
The steps under the request: method guides found and read, a power analysis and code run, then a literature search.

It read the power and design guides first

Steps appeared as soon as Priya sent it. Finding a method guide looked for guidance on cluster-randomised sample sizes, Reading a method guide opened the statistical-power and experimental-design guides, a Power Analysis step and a Ran code step did the arithmetic, and Searched PubMed looked for methods papers.

Method guides are analysis procedures Neutropic has reviewed in advance. Instead of answering from memory, it reads a vetted procedure and follows it. Click any step to see exactly what was read or run.

64 students per condition became 6 classes

The answer laid out three steps for equal-sized classes:

  1. Calculate the sample size for an individually randomised two-group comparison.
  2. Inflate it by the design effect: DEFF = 1 + (m − 1) × ICC, where m is students per class and ICC is the within-class correlation.
  3. Convert the inflated total into whole classes per condition.

Then came an example, clearly marked "Illustration, not a recommendation". Assuming 25 students per class, ICC = .05, d = .50, two-sided α = .05 and 80% power, the individual-level calculation gives 64 students per condition. The design effect is 2.2, so the rough adjustment gives 141 students per condition, or 6 classes per condition once rounded up.

The worked example: design effect 2.2, 141 students and 6 classes per condition, with numbered citations.
The worked example: design effect 2.2, 141 students and 6 classes per condition, with numbered citations.

An ICC of .05 sounds tiny, yet with 25 students per class it more than doubles the sample you need. That is why the answer stresses that the number of classes is what really counts.

Priya nearly copied "6 classes" straight into the proposal

This is where Priya nearly made a mistake. "6 classes per condition" looks like a finished answer. The answer itself says otherwise:

  • The approximation does not fully account for having only a few classes, and six per condition is not many.
  • Unequal class sizes can make simple inflation inaccurate. A simulation that matches the planned cluster-aware analysis is preferable then.
  • Take plausible ICC values from closely related studies or a pilot, and check more than one.

It was also honest about its sources: it found relevant records and abstracts, but the article pages could not be opened in that session, so it could not check their full methods or results. Priya decided to write the number down as an approximation and to plan a simulation check.

The sources came with it

The small numbers at the ends of sentences are citations. Open the References card and the 8 sources appear in a table, including chapters on cluster randomised trials, a paper on correcting ANOVA for clustering, ICC estimates for school-based studies and work on planning with unequal cluster sizes. Each row has a citation you can copy.

The References pane: 8 sources sorted by first citation, with Cite and DOI buttons.
The References pane: 8 sources sorted by first citation, with Cite and DOI buttons.

It suggested what to do next

At the bottom were two follow-ups:

  1. Explain how the plan should change if class sizes are unequal or few classes are available, including when simulation beats the design-effect shortcut
  2. Use the cited studies to build a study-specific design and sample-size plan, listing the assumptions still needed — Experimental Design agent

Click one and the conversation continues in the same chat. The Experimental Design agent can also be run from the Agent Gallery. It counts degrees of freedom in classes, keeps at least four classes per group and writes the sample-size rationale into a design report.

Next-step suggestions under the answer; the second is tagged Experimental Design.
Next-step suggestions under the answer; the second is tagged Experimental Design.

"How many students?" became "how many classes, and why?"

At the startNow
QuestionHow many students?How many classes, and on what basis?
MethodOrdinary two-group power analysisIndividual sample × design effect → whole classes
ExampleNone64 per condition → DEFF 2.2 → 6 classes per condition
CaveatsUnknownFew classes make the shortcut optimistic; simulate
SourcesNone8 cited methods sources

This is still an illustration built on assumed values. A real plan needs an ICC from similar studies or a pilot, actual class sizes and their spread, and expected attrition. Still, swapping "how many students?" for "how many classes?" fixes half the proposal.

This walkthrough used Pro. Lite (the free plan) gives the formula and the inputs you need; Expert also compares ICC scenarios and corrections for few clusters.

If whole classes are assigned, say so in your question. It changes the calculation from the start.

This week, ask with the one fact you know, just like Priya. Start with Neutropic for free

More from the blog