headline should name the category or name the pain, i only get 900 visits to test Launch Copy
Two candidates for the hero line. One names the category outright, the other leads with the problem in the user's words. Current landing page converts at about 4% to signup.
I expect roughly 900 visits across launch week. Running the numbers, detecting a two point lift at that baseline needs something like 1,500 per arm, so a split test during launch is theatre. Which do I pick on priors, and is there anything cheaper than an A/B test that actually tells me something?
@milkcratepress · 7mo ago · 3 replies
You already did the important work, which is realising you cannot test it. 900 visits at 4% is 36 conversions total. One noisy afternoon swamps any difference you would act on.
Pick on priors, and the prior is about whether the category exists in your visitor's head:
The part people miss: you are not choosing between the two lines. You are choosing the order. The other one is the subhead. Write both, put the category one wherever the visitor's eye lands first, and leave it for three months rather than five days.
Reply
Report
@seasonalpalette · 7mo ago
"You are choosing the order" reframed the whole thing for me. Category on top, pain sentence directly under it, done in ten minutes instead of a week of agonising.
Reply
Report
@goldenhourgus · 7mo ago
The five second test is worth more than a p-value at this volume. Show the page to five people for five seconds, take it away, ask what it does and who it is for. If three of them get it wrong, no headline test was going to save you.
Reply
Report