Hi, this is the Neutropic team!
Every exam season, some students can't sleep and feel their hands shake before a test. Mindfulness apps are everywhere now, so it is natural to wonder: would four weeks of app use actually reduce test anxiety? But the moment you try to test it properly, the questions pile up. What does the control group do? How many people do I need? When do I measure, and how do I analyse it? 🥲
Design decisions are hard to undo. Once the data are in, "Why this control group?" and "How did you justify the sample size?" are questions you can no longer fix.
In this post we follow Alex, an educational psychology master's student designing a thesis experiment. Alex is fictional, but every screen below comes from a real run in Neutropic.
Alex put five questions into one paragraph
Alex had not decided much: undergraduates from their own department, four weeks, test anxiety measured by questionnaire. They opened the Experimental Design agent from the Agent Gallery, chose Pro and sent this:
I want to design a study testing whether using a mindfulness app for four weeks reduces test anxiety in university students. I can recruit undergraduates from my department, and I plan to measure test anxiety with a questionnaire. Please lay out a suitable design, the control condition, how many participants I need, when to measure, and how to analyse it.
Two things make this a good request. It states what is already fixed (who, how long, how it is measured), and it lists the five answers it wants (design, control, sample size, timing, analysis). Anything left open is fine: as you will see, the agent writes it down as an assumption.
It turned the idea into questions and hypotheses, and listed its assumptions
The run went through eight steps in under two minutes. Step 1, Sharpen the question, set 3 research questions and 3 hypotheses. The primary hypothesis says that students assigned to the app will have lower post-intervention test-anxiety scores than the control group, and one research question explicitly adjusts for baseline scores.
Ten things the request did not say were recorded as assumptions: participants are undergraduates aged 18 or older who can consent; assignment is random; the comparator is an active control, a time-matched activity without mindfulness practice; the app condition is a recommended daily guided session. Alex now has a ready-made list to discuss with their supervisor.

It compared eight similar experiments before deciding
Step 2 found 84 papers in 9 databases and compared the designs of 8 experiments, reading some Method sections in full; the candidate list was saved as a table. Step 3 settled on a mixed design: 2 conditions (mindfulness app vs active control, between subjects) × 2 time points (baseline and post).
Step 4, Measures, was refreshingly honest. The plan named only "a validated self-report test-anxiety questionnaire", not a specific scale, so it could not be checked against Neutropic's instrument list and was flagged as needing its source confirmed. No invented scale name.
The sample size came from executed code: 98 people
Step 5 ran power-analysis code and proposed a total N of 98. The basis: the between-groups main effect of a mixed ANOVA (2 groups × 2 measurements, correlation between measurements ρ = 0.5), effect size f = 0.25, α = 0.05, power 0.80. The calculation code and the code that generates the random allocation schedule were attached as files, both marked as executed.
We reran the calculation with the same assumptions and got 98 too. 😎

Alex nearly copied 98 straight into the proposal
98 fits within the roughly 120 people Alex could recruit, so it was tempting to write it down and move on. But read the basis line again. Next to f = 0.25 it says an estimate not found in prior studies. Nobody yet knows whether the real effect is that large. By our own calculation, if the true effect is half the assumed size, 98 people give only about 30% power.
Two more details are worth a second look. First, the calculation is based on the group main effect, which averages baseline and post-test scores, while the primary hypothesis compares post-test scores adjusted for baseline. The two are related but not the same test, so it is worth recalculating for the analysis you will actually run. Second, 98 is the number analysed; no allowance for dropout was added. If you expect people to drop out, recruit more.
The report came with a "check first" list
Step 8 wrote the design report and a preregistration draft. The final answer summarises the study, the sample size, the assumptions and three things to check first:
- Choose and confirm the validated test-anxiety questionnaire: local-language suitability, scoring, licensing and administration format
- Specify the app, the session content and frequency, and how adherence will be recorded
- Specify the psychoeducational control materials and check they are time-matched to the app condition without mindfulness practice

Open the report and the This run section at the top shows the run level, what this level includes and the assumptions. A separate table compares reference designs. Save to notebook moves the report into a notebook where Alex can keep editing.

A vague idea became a design Alex can review
| At the start | Now | |
|---|---|---|
| Question | "Would an app reduce anxiety?" | 3 research questions, 3 hypotheses |
| Design | Undecided | Randomised 2 × 2 mixed design with a time-matched active control |
| Evidence | No idea what to read | 84 papers found, 8 experimental designs compared |
| Sample | No idea | 98 from executed code, with its assumptions spelled out |
| Next | Overwhelmed | Three things to check first and a preregistration draft |
To be honest about the limits: the questionnaire is still undecided, and the effect size is an assumption rather than a value from prior studies. The recruitable pool of about 120 is a default for in-person lab studies, not Alex's real number. Recheck the sample size for the analysis you plan, with a dropout margin, before you submit anything. Still, starting from a design with a clear list of open decisions is very different from starting from a blank page.
This walkthrough used Pro, which includes one alternative design and 8 reference studies. Lite (the free plan) gives the main design only, based on up to 5 reference abstracts. Expert adds an analysis plan, a manipulation check, a table of required N when the effect size is scaled from ×0.5 to ×1.25, an effect-size basis drawn from prior studies and one round of design review.
One paragraph about your experiment can get you to a design worth reviewing. Just recheck the sample-size assumptions.
This week, write down what you have decided and what you are unsure about, just like Alex. Start with Neutropic for free