How many observations does an experiment need?
Sample size depends on baseline rate or variance, MDE, alpha, power, allocation, attrition and clustering. It is calculated before the test and rounded up.
Direct answer
Size a test for a predefined minimum effect.
Sample size depends on baseline rate or variance, MDE, alpha, power, allocation, attrition and clustering. It is calculated before the test and rounded up.
n ≈ [z₁₋α/2√(2p̄(1−p̄)) + z₁₋β√(p₀(1−p₀)+p₁(1−p₁))]² / (p₁−p₀)²ESTIMAND
Estimation profile
- Unit of analysis
- Assigned unit × group × periodControlled scientific terminology
randomization unit per arm - Exact estimand
n per arm for a binary outcome, MDE, α and 1−β; equal allocation, normal approximation- Model or deliverable
n ≈ [z₁₋α/2√(2p̄(1−p̄)) + z₁₋β√(p₀(1−p₀)+p₁(1−p₁))]² / (p₁−p₀)²
01—08
Verifiable analysis framework
- 01Question
How many observations does an experiment need?
- 02Estimand
n per arm for a binary outcome, MDE, α and 1−β; equal allocation, normal approximation · randomization unit per arm
- 03Data
baseline_rate · target_rate · MDE · alpha · power · allocation · attrition · design_effect
- 04Model or deliverable
n ≈ [z₁₋α/2√(2p̄(1−p̄)) + z₁₋β√(p₀(1−p₀)+p₁(1−p₁))]² / (p₁−p₀)²
- 05Declared calculation
n ≈ [z₁₋α/2√(2p̄(1−p̄)) + z₁₋β√(p₀(1−p₀)+p₁(1−p₁))]² / (p₁−p₀)² → n per arm for a binary outcome, MDE, α and 1−β; equal allocation, normal approximation
- 06Uncertainty checks
Two-sided binary-outcome approximation · Equal allocation · Round upward and adjust for attrition and clustering
- 07Method-specific validation
Check data, estimate stability and interpretation limits. Two-sided binary-outcome approximation · Equal allocation · Round upward and adjust for attrition and clustering
- 08Limitations
Use observed post-hoc power as evidence.
Required variables
baseline_ratetarget_rateMDEalphapowerallocationattritiondesign_effect
Checks · Controlled scientific terminology
- Two-sided binary-outcome approximation
- Equal allocation
- Round upward and adjust for attrition and clustering
Size a test for a predefined minimum effect.
Use observed post-hoc power as evidence.
Scientific sources
1 source
Dataset · Tool
Method connections