Testing for Statisticians
Hypothesis testing is a framework for deciding whether data provide enough evidence against a null hypothesis. You compute a test statistic, find a p-value, and compare it to a significance level while watching Type I and Type II errors and power. Tests include t-tests, chi-square tests and nonparametric alternatives.
Stage: Inference · Part of the Statistician roadmap · Last reviewed 19 November 2026
What to learn in Testing
- State null and alternative hypotheses for a research question
- Run and interpret a two-sample t-test with its assumptions checked
- Interpret a p-value without treating it as the probability the null is true
- Calculate statistical power and the sample size needed for it
- Correct for multiple comparisons using Bonferroni or false discovery rate
Testing interview questions
What is a p-value?
A p-value is the probability of seeing data at least as extreme as what was observed, assuming the null hypothesis is true. A small value means the data are unusual under the null. It is not the probability that the null is true, nor the size of an effect.
What is the difference between Type I and Type II errors?
A Type I error rejects a true null hypothesis, a false positive, and its rate is the significance level alpha. A Type II error fails to reject a false null, a false negative, with probability beta. Power equals one minus beta, and lowering one error usually raises the other.
Where Testing fits in the Statistician roadmap
Testing belongs to the Inference stage (intermediate level). Learn from samples.
Other units in this stage
Back to the full Statistician roadmap