When alpha is 0.05, is a p-value of 0.01 more significant than a p-value of 0.04?

When alpha is 0.05, is a p-value of 0.01 more significant than a p-value of 0.04?

• A. Yes
• B. No
• C. Both are equally significant
• D. Only if effect size is large

Answer: (B)

The key idea is how p-values relate to a fixed alpha threshold. A p-value tells you the probability of getting data as extreme as observed if the null is true. Significance is decided by comparing that p-value to alpha. When alpha is 0.05, any p-value at or below 0.05 means the result is statistically significant. Here, both 0.01 and 0.04 are below 0.05, so both results reject the null at that level. In this sense, neither is “more significant” for the purpose of the decision: the test treats them as significant. The smaller p-value does indicate stronger evidence against the null, but that does not change the conclusion at the chosen alpha. The notion of needing a large effect size is unnecessary for declaring significance at this threshold.

The key idea is how p-values relate to a fixed alpha threshold. A p-value tells you the probability of getting data as extreme as observed if the null is true. Significance is decided by comparing that p-value to alpha. When alpha is 0.05, any p-value at or below 0.05 means the result is statistically significant. Here, both 0.01 and 0.04 are below 0.05, so both results reject the null at that level. In this sense, neither is “more significant” for the purpose of the decision: the test treats them as significant. The smaller p-value does indicate stronger evidence against the null, but that does not change the conclusion at the chosen alpha. The notion of needing a large effect size is unnecessary for declaring significance at this threshold.