Which Statistical Test Should I Use?

Quick decision tree

What is your primary question?
|
|-- Are you comparing a numeric or ordinal outcome between groups?
|   |-- Two groups
|   |   |-- Matched observations → Paired t-test / Wilcoxon signed-rank test
|   |   └-- Independent groups → Welch t-test / Wilcoxon rank-sum test
|   └-- Three or more independent groups → One-way ANOVA / Kruskal–Wallis test
|
|-- Are you studying the association between two variables?
|   |-- Two numeric or ordinal variables → Pearson or Spearman correlation
|   └-- Two categorical variables → Chi-square or Fisher's exact test
|
|-- Do you have repeated measurements over time for each participant or sample?
|   └-- Random-intercept linear mixed model
|
└-- Is the outcome time until an event, with possible censoring?
    └-- Kaplan–Meier curve; log-rank test to compare groups

Compare two groups

Two independent groups

Use Welch’s independent two-sample t-test for one numeric outcome in two separate groups when the question concerns a difference in means. It does not assume equal variances. Check independence, outliers, and whether a mean comparison is scientifically meaningful.

Use the Wilcoxon rank-sum (Mann–Whitney U) test for a rank-based comparison of two independent groups with a numeric or ordinal outcome. It compares distributions through ranks; a significant result should not automatically be described as a difference in medians unless the distributional context supports that interpretation.

Two matched measurements

Use a paired t-test when each observation is a correctly matched pair, such as before-and-after measurements from the same participant. The analysis concerns the within-pair differences, so review the distribution and outliers of those differences.

Wilcoxon signed-rank is a rank-based paired alternative. Its usual location-shift interpretation assumes that nonzero paired differences are symmetrically distributed; zero differences are not ranked.

Compare three or more groups

One-way ANOVA

Use one-way ANOVA for one numeric outcome in three or more independent groups when the question is whether group means differ. A significant overall ANOVA indicates evidence that at least one mean differs, but follow-up comparisons are needed to identify where differences lie. Review independence, outliers, within-group distributional shape, and variance homogeneity. When variances differ importantly, consider a method designed for unequal variances, such as Welch ANOVA.

Kruskal–Wallis test

Use Kruskal–Wallis for three or more independent groups with a numeric or ordinal outcome when a rank-based comparison is appropriate. It tests for differences among group rank distributions. Dunn pairwise comparisons with a multiple-testing adjustment are one possible follow-up after a significant overall test.

Related resources

For a closer look at choosing between a two-sample t-test and one-way ANOVA for independent groups, read T-test vs. ANOVA: How to Choose for Comparing Group Means. For the design distinction between matched and separate observations, read Paired vs. Independent Samples: How Study Design Changes Your Statistical Test. For choosing a correlation summary, read Pearson vs. Spearman Correlation: Which Measure of Association Should You Use?. For choosing a test for two categorical variables, read Chi-square vs. Fisher’s Exact Test: How to Choose for Categorical Data. For an introduction to distributional shape and diagnostics, read What Is a Normal Distribution? A Practical Guide for Research Data.

Study the relationship between two variables

Pearson correlation

Use Pearson correlation to describe a linear association between two numeric variables. Inspect a scatterplot first: curvature, influential outliers, or non-independent observations can make a correlation misleading. Correlation does not establish causation or adjust for confounding.

Spearman correlation

Use Spearman correlation when the question concerns a monotonic rank association between two ordinal or numeric variables. It can be useful for a monotonic relationship that is not linear, but it does not solve problems caused by non-independence, outliers, or confounding.

Analyze two categorical variables

Chi-square test of independence

Use a chi-square test when both variables are categorical and each observation belongs in one contingency-table cell. Observations should be independent, categories mutually exclusive, and expected cell counts checked before relying on the chi-square approximation.

Fisher’s exact test

For a 2×2 contingency table, Fisher’s exact test is particularly useful when expected counts are small and the chi-square approximation may be unreliable. The appropriate approach for larger tables depends on table size, study design, and the available computational method.

Repeated measurements and time-to-event outcomes

Random-intercept linear mixed model

Use a random-intercept linear mixed model when the same participant, animal, or sample is measured repeatedly over time and the outcome is numeric. A common specification includes a random intercept for the participant or sample ID and fixed effects for treatment group, time, and their interaction. More complex designs may require different random-effects or covariance structures.

Kaplan–Meier curve and log-rank test

Use Kaplan–Meier methods when the outcome is time until an event and some observations may be right-censored. The curve estimates survival over time; the log-rank test compares survival curves across groups. Define the time origin and event clearly, consider the assumption of non-informative censoring, and remember that these methods do not adjust for covariates.

When to seek additional statistical support

This guide is a useful starting point for common research questions, but some designs need a tailored analysis plan. Consider consulting a statistician before choosing an analysis if your study involves clustering, repeated treatments, multiple outcomes, substantial missing data, covariate adjustment, a complex survival or longitudinal design, or a pre-registered analysis plan.

Before you run any test

1. Define the estimand and primary outcome before examining the result.
2. Confirm whether observations are independent, paired, or repeated.
3. Inspect the data and plot it; no test output replaces data-quality checks.
4. Report effect sizes and confidence intervals where available, not only p-values.
5. Account for multiple comparisons when making many inferences.