P-Value Calculator — Free Online Hypothesis Testing Tool

How to Use the P-Value Calculator

1. Enter your test statistic: Input the z-score or test statistic from your hypothesis test. This value typically comes from a z-test, and it represents how far your observed result is from the null hypothesis in standard units.
2. Select the test type: Choose "Two-Tailed" when testing for any difference from the null hypothesis (most common). Choose "Left-Tailed" when testing specifically whether a parameter is less than the hypothesized value. Choose "Right-Tailed" when testing whether a parameter is greater than the hypothesized value.
3. Read the p-value: The p-value appears prominently at the top of the results panel, calculated to six decimal places. A smaller p-value indicates stronger evidence against the null hypothesis.
4. Check significance at common levels: The results automatically compare your p-value against three common significance levels (α = 0.01, 0.05, and 0.10), clearly indicating whether you should reject or fail to reject the null hypothesis at each level.

The calculator uses the standard normal (z) distribution for probability calculations. All results update in real time as you modify inputs. The interpretation guide provides a plain-language summary of the evidence strength.

P-Value Formulas

Variables Explained

• p: The p-value — the probability of observing a test statistic at least as extreme as the one calculated, under the null hypothesis.
• z: The test statistic (z-score), representing how many standard deviations the observed result is from the null hypothesis value.
• Φ(z): The cumulative distribution function (CDF) of the standard normal distribution, giving P(Z ≤ z).
• α (alpha): The significance level — the threshold below which the p-value is considered statistically significant. Common values are 0.05, 0.01, and 0.10.

Step-by-Step Example

A researcher conducts a two-tailed z-test and obtains a test statistic of z = 2.35. Find the p-value and determine significance.

1. Identify the test statistic: z = 2.35
2. Find the CDF value: Φ(2.35) = 0.9906
3. For a two-tailed test: p = 2 × min(0.9906, 1 - 0.9906) = 2 × 0.0094 = 0.0188
4. Compare to α = 0.05: Since 0.0188 < 0.05, the result is statistically significant
5. Compare to α = 0.01: Since 0.0188 > 0.01, the result is not significant at this stricter level

At the 5% significance level, we reject the null hypothesis. The observed test statistic of 2.35 would occur only about 1.88% of the time if the null hypothesis were true, providing strong evidence against it.

Practical Examples

Example 1: Angela's A/B Test for Website Conversion

Angela is testing whether a new website design (B) increases conversion rates compared to the original (A). The original has a 3.2% conversion rate. After 10,000 visitors to design B, she observes a 3.8% conversion rate. Her z-test gives a test statistic of z = 2.45. She uses a right-tailed test since she is testing for an increase.

• Test statistic: z = 2.45 (right-tailed)
• P-value: 1 - Φ(2.45) = 1 - 0.9929 = 0.0071
• Significant at α = 0.05: Yes
• Significant at α = 0.01: Yes

With a p-value of 0.0071, the improvement in conversion rate is highly statistically significant. Angela can confidently implement design B, knowing there is less than a 1% chance that the observed improvement is due to random variation alone.

Example 2: Professor Wang's Drug Trial Analysis

Professor Wang is testing whether a new medication lowers cholesterol more than a placebo. The z-test comparing the treatment group to the placebo group yields z = -1.82. He uses a left-tailed test (testing for a decrease in cholesterol).

• Test statistic: z = -1.82 (left-tailed)
• P-value: Φ(-1.82) = 0.0344
• Significant at α = 0.05: Yes
• Significant at α = 0.01: No

The p-value of 0.034 is below the 0.05 threshold, suggesting the drug does reduce cholesterol more than the placebo. However, it does not pass the stricter 0.01 threshold. Professor Wang may recommend a larger follow-up study to confirm the effect with greater confidence, especially given the medical implications of the finding.

Example 3: Rachel's Quality Assurance Testing

Rachel tests whether the average weight of cereal boxes differs from the labeled 500g. Her sample of 50 boxes yields a test statistic of z = 0.87. She uses a two-tailed test since the boxes could be over or under weight.

• Test statistic: z = 0.87 (two-tailed)
• P-value: 2 × (1 - 0.8078) = 2 × 0.1922 = 0.3844
• Significant at α = 0.05: No
• Significant at α = 0.10: No

With a p-value of 0.384, Rachel fails to reject the null hypothesis. The average box weight does not significantly differ from 500g. She reports that the production line is operating within acceptable tolerances, and no corrective action is needed. For a more detailed look at the data spread, she might use our confidence interval calculator.

P-Value Interpretation Reference Table

Tips and Complete Guide

Understanding Type I and Type II Errors

A Type I error (false positive) occurs when you reject a true null hypothesis. The probability of this is α, your significance level. A Type II error (false negative) occurs when you fail to reject a false null hypothesis. The probability of this is β, and statistical power (1 - β) is the probability of correctly detecting a real effect. Reducing α increases the risk of Type II errors, creating a fundamental tradeoff. Choose α based on which error is more costly in your specific context.

Effect Size and Practical Significance

A p-value alone does not tell you how large or important an effect is. A statistically significant result (p < 0.05) with a tiny effect size may be practically meaningless, especially with large samples. Always report effect sizes alongside p-values. Cohen's d measures the standardized difference between means: d = 0.2 is small, d = 0.5 is medium, and d = 0.8 is large. For our sample size calculator, you can determine how many observations you need to detect a specific effect size.

Multiple Testing and Correction Methods

When testing multiple hypotheses simultaneously (such as comparing 10 groups), the probability of at least one false positive increases dramatically. With 20 independent tests at α = 0.05, there is about a 64% chance of at least one false positive. The Bonferroni correction divides α by the number of tests (0.05 / 20 = 0.0025). The Holm-Bonferroni method is a less conservative alternative that maintains better power while controlling the family-wise error rate.

Common Mistakes to Avoid

• Interpreting p-value as the probability H₀ is true: A p-value of 0.03 does not mean there is a 3% chance the null hypothesis is true. It means there is a 3% chance of observing data this extreme if the null hypothesis were true. This distinction is fundamental but often confused.
• Treating p = 0.049 and p = 0.051 as fundamentally different: Statistical significance is not a binary cliff. A p-value of 0.049 is essentially the same strength of evidence as 0.051. Avoid the temptation to celebrate one while dismissing the other. Report exact p-values and let readers judge the evidence.
• Using one-tailed tests to get smaller p-values: Choosing a one-tailed test after seeing the data direction is a form of p-hacking. The decision between one-tailed and two-tailed tests must be made before data analysis based on your research question, not based on what makes the result significant.
• Confusing "not significant" with "no effect": Failing to reject the null hypothesis does not prove the null hypothesis is true. It only means you lack sufficient evidence to reject it. The difference could be real but too small for your sample to detect. Absence of evidence is not evidence of absence.
• Ignoring effect size with large samples: With 100,000 observations, virtually any non-zero difference becomes statistically significant. Always pair your p-value with an effect size measure to assess practical importance.

Frequently Asked Questions