How to Interpret P-Values and Confidence Intervals Together

The short answer

Start with the estimate

Before looking at a p-value, ask: what is the estimated effect? The answer depends on the question and outcome.

• For two group means, it may be a mean difference.
• For a binary outcome, it may be a risk difference, risk ratio, or odds ratio.
• For an association, it may be a correlation coefficient or regression coefficient.
• For time-to-event data, it may be a hazard ratio or a difference in survival probability at a meaningful time point.

The estimate gives the result a scale and direction. If a treatment group’s mean outcome is 4.2 units lower than control, the estimated difference is −4.2 units. Whether that difference is large, small, beneficial, or meaningful depends on the units, participants, and a pre-specified scientific or clinical benchmark.

What a p-value adds

Suppose a pre-specified comparison reports a mean difference of −4.2 units with p = 0.009. The p-value asks how compatible the observed data are with a null model of no population mean difference, assuming the model and analysis conditions are correct.

It does not tell you the probability that the null hypothesis is true, the probability that the treatment works, the size of the treatment effect, whether the effect matters to patients or decision-makers, or whether the study design ruled out bias or confounding.

P-values are most useful when tied to a pre-specified question and interpreted as one part of the evidence rather than as a pass/fail label for an entire study.

What a confidence interval adds

A confidence interval places the estimate in context. A 95% confidence interval is produced by a procedure that, across many comparable samples and under the model assumptions, would contain the true parameter 95% of the time.

For one completed study, it is better not to say there is a 95% probability that the fixed true value lies inside the calculated interval. The interval is one result from a repeated-sampling procedure. In practice, it shows which effect sizes are reasonably compatible with the data and model.

For example:

Mean difference: −4.2 units, 95% CI −7.3 to −1.1, p = 0.009

The estimate favors a lower mean outcome in the treatment group. The interval suggests that effects from a 1.1-unit to a 7.3-unit reduction are compatible with the data and model. The p-value provides evidence against a null difference of zero.

Read the estimate, interval, and p-value together

The null value depends on the effect measure: it is 0 for a difference or correlation and 1 for a ratio such as a risk ratio, odds ratio, or hazard ratio.

Why 95% intervals and 0.05 thresholds often agree

For a matching two-sided hypothesis test and two-sided 95% confidence interval, an interval that excludes the null value usually corresponds to a p-value below 0.05. An interval that includes the null value usually corresponds to a p-value of 0.05 or greater.

This connection is useful, but it should not turn the interval into another threshold test. The distance from the null value, the range of values in the interval, and the meaning of those values are more informative than whether one boundary barely crosses zero or one.

A result with p = 0.049 and an interval ending at −0.01 is not categorically different from one with p = 0.051 and an interval ending at 0.01. Both may be uncertain, sensitive to analytical choices, and insufficient to explain practical importance by themselves.

Precision is not the same as importance

Large samples can estimate small effects very precisely. That can produce a narrow confidence interval and a small p-value for a difference that has little scientific or clinical relevance.

Small samples can produce a large point estimate with a wide confidence interval. The result may look promising, but the interval may still include negligible, harmful, or very large effects. A small p-value is not a substitute for precision, and a large p-value is not proof that an effect is absent.

When possible, define a smallest effect size of interest before analysis. This might be the minimum clinically important difference, a biologically meaningful change, or a decision-relevant threshold. Interpret the confidence interval in relation to that benchmark, not only the null value.

Common mistakes

“The p-value tells me whether the result is important.”

No. The p-value does not measure the size, direction, practical importance, or clinical importance of an effect. Use the estimate, its units, a confidence interval, and a pre-specified meaningful-effect benchmark.

“The true effect has a 95% chance of being inside this 95% interval.”

Not in the usual frequentist interpretation. The true parameter is fixed; the interval procedure has the stated long-run coverage under its assumptions. The interval is still a practical display of the values compatible with the data and model.

“Non-overlapping 95% confidence intervals prove a difference between groups.”

Not necessarily. The overlap of two separate confidence intervals is not a direct test of the difference between their estimates. Calculate and report the confidence interval for the comparison that answers the study question.

“If the p-value is above 0.05, I should report no effect.”

No. Report the estimate and confidence interval. A larger p-value can reflect an imprecise study rather than a precisely estimated null effect.

A practical reporting template

“Compared with control, the treatment group had a mean outcome 4.2 units lower (95% CI −7.3 to −1.1; Welch two-sample t-test, p = 0.009). The confidence interval includes reductions from 1.1 to 7.3 units; the clinical relevance of this range should be judged against the pre-specified meaningful difference.”

The wording should match the outcome and design. For ratio measures, state the reference group and whether a value above or below 1 represents higher risk. For adjusted models, state the covariates and the model used.

When to seek additional statistical support

Seek additional support when intervals are based on complex samples, clustered or repeated observations, sparse events, multiple comparisons, missing data, nonstandard estimands, or model-dependent transformations. Also seek support when the result will guide a clinical, regulatory, or causal decision. The interval and p-value are only as trustworthy as the design, data, and method that produced them.

Related resources

For the broader framework of null hypotheses, alternatives, thresholds, and error rates, see How Hypothesis Testing Works: A Practical Guide for Research. For a common distributional diagnostic before a mean-based analysis, see What Is a Normal Distribution? A Practical Guide for Research Data. For study planning around error rates, power, and sample size, see Type I and Type II Errors, Power, and Sample Size.