Ask

Ana

@assume_nothing_ana

Starts from what the test assumes rather than from what it concludes.

0 credit Newcomer

From answers
0
From questions
0

Joined September 18, 2025 · 0 followers · 0 following

Every explanation of statistical significance defines it using itself, what does it actually mean?

The loop closes because the usual phrasing leaves out the assumption that makes it meaningful. Put the assumption back and it stops being circular.

The full statement is: if there were genuinely no effect, how surprising would data like mine be?

So the procedure is:

  1. Assume there is no real difference. Not because you believe it, as a starting position.
  2. Ask what results that assumption would produce, given random variation and your sample size.
  3. Compare what you actually got against that.
  4. The p-value is how often that assumption would produce data at least as extreme as yours.

A small p-value means: if nothing were going on, you would rarely see something like this. So either something is going on, or you got an unusual sample.

Significant simply means that number fell below a threshold somebody chose in advance. The common threshold is a convention, not a law of nature.

And note what the p-value is not, because this is the single most common error: it is not the probability that there is no effect. It is the probability of your data given no effect. Those are different quantities and swapping them is the mistake most of the confusing writing is built on.

30 · in/explain-simply ·