Forty signups a month and every A/B test comes back significant, I do not believe any of them

Your instinct is right and the arithmetic backs it up: at eleven versus seven you cannot detect anything, and the tool telling you otherwise is the tool being used outside its range.

The short version of why. To detect a difference you need enough events that ordinary randomness cannot produce the gap you are seeing. With single-digit conversion counts, randomness produces gaps like yours constantly - flip a coin eighteen times and you will regularly get eleven and seven, and nobody would call that a biased coin.

Two specific traps you have probably also hit:

Peeking. If you check the result as it accumulates and stop when it looks significant, you will find significance almost every time, in noise, guaranteed. Sequential looking at a running test breaks the statistics completely, and every dashboard invites it.

Small effects need enormous samples. People imagine a test detects a ten percent improvement. Detecting that reliably needs sample sizes in the thousands per variant. At forty signups a month you would be waiting years, not a year.

So the honest position: at your size, A/B testing is not a tool you own yet. That is not a failure, it is a scale fact. The good news is that the alternatives are better at this size anyway, and both of the answers below are things you can do this week.

What to do instead, and it is much higher return at forty signups a month: talk to the people.

At your volume you can contact every single signup. Not a survey - a short direct message asking what they were trying to do and what nearly stopped them. Seven or eight real conversations will tell you more about your landing page than any test you can run this year, because they tell you why, which no amount of conversion data ever does.

What to ask, roughly:

• What were you trying to solve when you found this
• What did you expect it to do
• Was there a moment where you nearly did not sign up
• What did you use before

The answers cluster fast. If four of seven say they could not tell what it did, that is a finding with a clear action, and it is far more actionable than a sixty percent lift you cannot reproduce.

Also watch a few people use it, if they will let you. Five sessions of watching someone attempt your onboarding uncovers things no metric surfaces, because a person getting stuck for ninety seconds and then succeeding looks identical in your funnel to a person who sailed through.

This is unscalable and that is fine: it is the right tool at forty a month, exactly as hand-written emails are the right acquisition channel at that size.

The other half: when you do change things, change them big enough that you do not need statistics.

A test exists to detect a difference too small to see. At your size, the changes worth making are the ones that are not subtle: a completely different positioning, removing a required field, cutting three steps out of onboarding, changing who the page is addressed to. If a change is real it will show up in the monthly numbers without a significance calculation, and if it needs a calculator to see, it was not worth your month.

How to run that honestly without a testing tool:

Change one thing, and note the date. Compare a month before against a month after, and look at the direction and the size rather than a p-value.

Watch a longer window than feels necessary, because your monthly numbers are noisy too. One good month is not evidence.

Write down what you expected before you ship. This is the single most useful discipline available at small scale, it stops you from reinterpreting whatever happened as a success, which is very easy to do with forty data points.

And track the whole funnel rather than the one number. A headline that raises signups and lowers activation has made things worse, and at your size that trade is common and easy to miss.