Why use p-hat = 0.5 as the default?

The product p-hat(1 - p-hat) is maximized at p-hat = 0.5, giving the largest (most conservative) sample size. If you have a reasonable prior estimate, use it to get a smaller, more efficient n. For example, p-hat = 0.1 reduces the required n by about 64%.

How does the margin of error affect sample size?

Sample size is inversely proportional to E squared. Halving the margin of error (e.g., from 6% to 3%) quadruples the required sample size. This is why ultra-precise polls are very expensive.

Should I apply a finite population correction?

If sampling more than 5% of the population, apply FPC: n_adj = n / (1 + (n-1)/N), where N is the population size. For large populations (N > 20n), FPC makes negligible difference.

What is statistical power and why does 80% matter?

Power is the probability of correctly rejecting a false null hypothesis (detecting a real effect). 80% is the conventional minimum: it means a 20% chance of missing a real effect. Some fields (e.g., clinical trials) use 90% power for critical decisions.

One-sided vs. two-sided test: which should I use?

Use two-sided unless you have strong prior knowledge that the effect can only go one direction. One-sided tests require smaller samples but are harder to justify. Most journals and regulators expect two-sided tests.

What is effect size and how do I choose delta?

Effect size = delta / sigma. Cohen suggests d = 0.2 (small), 0.5 (medium), 0.8 (large). Choose delta based on the minimum clinically or practically meaningful difference, not what you expect to find.

What is a Minimum Detectable Effect (MDE)?

MDE is the smallest improvement you want your test to reliably detect. Smaller MDEs require exponentially more samples. For a 3% baseline, a 10% relative MDE (0.3% absolute) is very hard to detect; a 20% relative MDE (0.6% absolute) is more practical.

How do I estimate test duration?

Divide total required sample size by your daily traffic allocated to the test. For example, 100,000 total visitors / 2,000 per day = 50 days. Always run for at least 1-2 full business cycles (1-2 weeks minimum) to account for day-of-week effects.

Should I use one-tailed or two-tailed for A/B tests?

Most A/B testing platforms use two-tailed tests by default (they detect both improvements and regressions). This is recommended because a change could unexpectedly hurt performance. One-tailed tests reduce required sample size by about 20% but miss negative effects.

What if my baseline conversion rate is very low (< 1%)?

Low baselines drastically increase required sample sizes. For a 0.5% baseline with 20% relative MDE, you need approximately 500,000 visitors per variant. Consider using a larger MDE or a different metric (e.g., revenue per visitor) that has higher variance.

All Tools

Sample Size Calculator

Calculate the required sample size for surveys, A/B tests, and experiments with power analysis, step-by-step solutions, and Python code.

Enter Parameters

Confidence Level

Margin of Error (E)

Population Std Dev (sigma)

Sample Data:

Enter your parameters and click Calculate n.

Quick Reference:

90% CI: z* = 1.645
95% CI: z* = 1.960
99% CI: z* = 2.576
80% Power: z_beta = 0.842
90% Power: z_beta = 1.282

Frequently Asked Questions

What if I do not know the population standard deviation?

Use an estimate from a pilot study, prior research, or the range/4 rule (sigma approximately equals range / 4). If completely unknown, run a small pilot study of 20-30 observations to estimate sigma.

Why do we always round up the sample size?

Rounding down would give a margin of error slightly larger than desired. Rounding up ensures the actual margin of error is at most E. For example, n = 216.09 is rounded to 217.

Does a larger confidence level require more samples?

Yes. Going from 95% to 99% confidence increases z* from 1.960 to 2.576, which increases the required n by about 73%. Higher confidence demands wider intervals, requiring more data to maintain the same margin of error.

Related Tools

Confidence Interval

CI for means and proportions

T-Test Calculator

One-sample, two-sample & paired t-tests

Chi-Square Calculator

Independence & goodness of fit