Ask
74
@seasonalpalette ·

headline should name the category or name the pain, i only get 900 visits to test Launch Copy

Two candidates for the hero line. One names the category outright, the other leads with the problem in the user's words. Current landing page converts at about 4% to signup.

I expect roughly 900 visits across launch week. Running the numbers, detecting a two point lift at that baseline needs something like 1,500 per arm, so a split test during launch is theatre. Which do I pick on priors, and is there anything cheaper than an A/B test that actually tells me something?

8 answers Share
Report

Answering anonymously — a moderator will review it first.

  • @milkcratepress · 7mo ago · 3 replies

    You already did the important work, which is realising you cannot test it. 900 visits at 4% is 36 conversions total. One noisy afternoon swamps any difference you would act on.

    Pick on priors, and the prior is about whether the category exists in your visitor's head:

    • if people already search for the category by name, lead with the category. A stranger needs to know what bucket you are in inside two seconds, and being clever costs you that
    • if the category does not exist yet, lead with the pain, because there is no word for you to use

    The part people miss: you are not choosing between the two lines. You are choosing the order. The other one is the subhead. Write both, put the category one wherever the visitor's eye lands first, and leave it for three months rather than five days.

    58
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
    • @seasonalpalette · 7mo ago

      "You are choosing the order" reframed the whole thing for me. Category on top, pain sentence directly under it, done in ten minutes instead of a week of agonising.

      19
      Share
      Reply

      Answering anonymously — a moderator will review it first.

      Report
    • @goldenhourgus · 7mo ago

      The five second test is worth more than a p-value at this volume. Show the page to five people for five seconds, take it away, ask what it does and who it is for. If three of them get it wrong, no headline test was going to save you.

      12
      Share
      Reply

      Answering anonymously — a moderator will review it first.

      Report
  • @sdcard_sid · 7mo ago

    Steal the phrasing rather than inventing it. Read your last twenty support messages, DMs, or whatever people wrote in your waitlist form, and find the noun they keep using. That noun is your headline. Nobody types your category name if they do not know your category exists, and nobody recognises your pain sentence if it is phrased the way a founder would phrase it.

    27
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
  • @goldenhourgus · 7mo ago · 2 replies

    Run them sequentially instead. A for the first half of launch week, B for the second. Yes it is confounded, but you will pick up twenty qualitative signals from how people describe the product back to you, and that beats a broken split test.

    34
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
    • @seam_ripper_joy · 7mo ago

      Traffic composition changes hour to hour during a launch — day one is aggregators, day three is people who heard about it secondhand. A sequential test across those two populations is close to worthless. Fine idea on steady-state traffic, bad idea in launch week specifically.

      16
      Share
      Reply

      Answering anonymously — a moderator will review it first.

      Report
  • @blistermaps · 7mo ago

    Both, literally. <h1> names the category, the paragraph under it names the pain. That is not a compromise, that is just what a hero section is supposed to be.

    18
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
  • @pomodoro_pat · 7mo ago

    900 visits is not a sample, it is an anecdote with a spreadsheet attached.

    9
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report