Ask
26
@chapel_lane ·

In a content product the content is the product, and one wrong answer is the whole review

Test prep apps came out of the review pull with an ordinary looking profile: price 23, accuracy 13, ads 7 across 44 low star reviews. Then I read the accuracy ones and they are not ordinary at all, because of what the product is for.

"I was getting ninety five percent of the questions right in the app and I failed the real test. Several topics are not in the app at all."

"Studied for the motorcycle licence for weeks. Not one of the questions I studied came up."

"The theory section was incomplete, and in the practice test wrong information was marked as the correct answer."

A bug in most products costs somebody an afternoon. Here it costs a test fee, a day off work and a retake, and the review is written by somebody who has just failed something. There is no severity below that in this category.

Two more things in the same pile that are decisions rather than bugs. Different licence classes share one generic question pool, so people studying for a truck or a motorcycle licence get car questions and say so. And the second language is behind the paywall, which in this category means the people most likely to fail the written test are the ones being charged extra to study in a language they read.

So the first version here is not an app with some questions in it. It is the current official question set, split correctly by class, and everything else can wait.

8 answers Share
Report

Answering anonymously, a moderator will review it first.

  • @quiet_carriage · 2w ago

    Careful with the ninety five percent quote though. Someone who fails a test is highly motivated to blame the app, and a practice tool that drills the same hundred questions will always produce inflated confidence regardless of whether the content is right. Both stories fit that review.

    15
    Share
    Reply

    Answering anonymously, a moderator will review it first.

    Report
  • @kiln_yard · 2w ago · 4 replies

    The uncomfortable part is that content decays on somebody else's schedule. The regulator changes the question set and your perfect app is quietly wrong from that morning, with no deploy on your side and no error in your logs. That is an ongoing job, not a launch task.

    20
    Share
    Reply

    Answering anonymously, a moderator will review it first.

    Report
    • @chapel_lane · 2w ago · 3 replies

      Which is the actual moat, I think. Anyone can build the quiz UI in a weekend. Almost nobody will still be checking the official source every month in year three, and the reviews tell you exactly who stopped.

      16
      Share
      Reply

      Answering anonymously, a moderator will review it first.

      Report
      • @kiln_yard · 2w ago

        Put a visible last verified date on the question set and you have turned your maintenance burden into the thing people choose you for.

        11
        Share
        Reply

        Answering anonymously, a moderator will review it first.

        Report
      • @two_bulbs · 2w ago

        Does anyone actually publish that date? I have never seen one.

        9
        Share
        Reply

        Answering anonymously, a moderator will review it first.

        Report
  • @chapel_lane · last wk.

    Fair, and on its own I would discount it. The one I cannot explain away is wrong information marked as the correct answer, because that is checkable and it does not depend on how the person felt afterwards.

    1
    Share
    Reply

    Answering anonymously, a moderator will review it first.

    Report
  • @fen_road · last wk.

    Putting the second language behind the paywall is the detail I would fix first. It is a small revenue line and it is aimed precisely at the people with the hardest time passing, which is a thing you have to be able to say out loud.

    1
    Share
    Reply

    Answering anonymously, a moderator will review it first.

    Report
  • @oak_desk · last wk.

    Ads in the middle of a practice exam. Someone measured the revenue and not the score.

    1
    Share
    Reply

    Answering anonymously, a moderator will review it first.

    Report