I can measure how long a new account takes to reach the thing the product is actually for (median about 26 minutes), and I can measure the gap between someone's first and second session. I do not have the volume to instrument and act on both properly, so I want to pick one. For anyone who has watched a cohort for a year: which of those two told you something you could actually change?
In our data the return gap was the sharper split by a distance - accounts that came back within about two days behaved completely differently from ones that took a week, and the boundary was visible without any statistics. Time to first value was noisy because half our signups did the setup in one long session and half did it across three sittings around their actual job, so the clock was measuring their calendar rather than our product.
The clock measuring their calendar is a good way to put it. We ended up counting sessions to first value instead of minutes, which removed most of the noise.
Both of those are downstream of who signed up. When I finally segmented by acquisition source, the differences between sources were larger than anything either time metric was showing me, and half my supposed onboarding problem was one channel sending people who were never going to use it. Segment first or you will optimise an average that describes nobody.
Fair, though at forty or fifty accounts the segments get too small to compare. Worth doing as soon as any single source produces enough people to look at on its own.
Instrument both, act on neither for a month, and go and ask five people who did not come back. The metric tells you where, the phone call tells you why.
I ground the median time to first value down substantially over a quarter - removed steps, added defaults, skipped the tour - and retention did not move at all. What I had actually done was help people who were already going to succeed do it slightly faster. The number improved and the business did not, which was a useful and annoying lesson.
@csv_apologist · 5mo ago
I have watched both for about two years on a small B2B tool. Time to first value was the more actionable one because it points at a specific part of the product - when it went up, it was always a concrete thing, an added form field, a slow import, a confusing empty state. The gap between sessions was the better predictor of who stayed but it told me nothing about what to fix, because by the time it was long the person had already decided. Measure the second, act on the first.
Reply
Report