five second test: four of six readers said my page "does automation" and that is not what it does
Showed the homepage to six people for five seconds each, then asked what the product does. Four said some version of "automates something", one said "analytics dashboard", one said "I did not read it, I looked at the screenshot".
The product schedules and confirms appointments for small clinics. The headline currently says "Stop losing hours to admin". Which, reading it now, could be literally any software ever made.
I get about 400 visits a month so I cannot A/B my way out of this. How do you actually fix a headline when you have no traffic to test with?
@stipend_sam · 4mo ago · 2 replies
On the testing question specifically: at 400 visits a month you cannot run a copy test and you should stop feeling bad about it.
A rough rule for how many visitors per variant you need is sixteen times p times one-minus-p, divided by the square of the difference you want to detect. If you convert at 2% and you want to be confident about a move to 3%, that is somewhere around three to four thousand visitors per variant. You would need most of a year and by then you would have changed six other things.
So do qualitative until you have volume. Six five-second tests told you something real in an afternoon. Do twelve more with different people, change one thing, do twelve more. That loop is faster and more honest than a test you cannot power.
Reply
Report
@trace_tilly · 4mo ago
And when you do have volume, test things that are big enough to matter. A headline rewrite that changes who the page is for can move conversion by a multiple. Swapping two adjectives moves it by an amount you will never be able to measure at any traffic level a solo product sees.
Reply
Report