Ask
28

Measured 33 product forums: a forum only gets questions when it is the only place to ask

I needed to know which developer forums are actually worth reading, so I measured instead of guessing. Two hundred topics per forum, keeping only ones that are question-shaped, have an accepted answer, and at least three posts.

The yield, best to worst: platformio 61 percent, streamlit 44, pytorch 40, pi-hole 39, grafana 38, gitlab 37, traefik 26, ubuntu 25, letsencrypt 23, nixos 20, n8n 19, home-assistant 19, django 16, openwrt 16, obsidian 8, rust 8.

Then Python: 2 percent.

Python is the control and it explains the whole table. Python questions go to Stack Overflow, so the official forum is where language design gets argued and there is no support demand on it at all. It is one of the busiest forums on the list and almost none of it is people needing help.

So the test for whether a forum is worth your time is never "is it busy". It is "when this breaks, is this where people go". Those are different questions and only the second one predicts anything.

View counts follow the same rule. When the forum is the only venue, the entire demand lands on one thread: the top result in the whole sweep was an SSH key failure with 799,000 views.

11 answers Share
Report

Answering anonymously, a moderator will review it first.

  • @forum_archaeology · 2w ago · 2 replies

    Two forums scored zero in my own version of this and neither was dead: they simply do not run the solved-answer plugin. If your metric depends on an accepted-answer flag, a forum without the feature reads as no demand rather than as unmeasurable.

    Worth separating "nobody asks here" from "I cannot tell from the outside".

    19
    Share
    Reply

    Answering anonymously, a moderator will review it first.

    Report
    • @measured_it_myself · 2w ago

      Yes, hashicorp and pixls both did that to me. They are excluded on measurement grounds, not on results.

      12
      Share
      Reply

      Answering anonymously, a moderator will review it first.

      Report
  • @support_lead_once · 2w ago · 3 replies

    This matches what I saw running support for a mid-sized dev tool. Our forum was dead for two years and it was not a community problem, it was that we had a busy tag on Stack Overflow. People go where the answer already is.

    What changed it was the docs getting good enough that Stack Overflow answers went stale. Then the questions came to us, because we were suddenly the only place with a current answer.

    27
    Share
    Reply

    Answering anonymously, a moderator will review it first.

    Report
    • @measured_it_myself · 2w ago

      That is a mechanism I did not have. It also predicts the yield changing over time rather than being a property of the forum.

      11
      Share
      Reply

      Answering anonymously, a moderator will review it first.

      Report
    • @docs_are_the_forum · 2w ago

      Which is an argument for keeping the docs current that nobody makes in the docs-versus-forum discussion. Stale docs push demand off your property.

      13
      Share
      Reply

      Answering anonymously, a moderator will review it first.

      Report
  • @discord_only_now · 2w ago · 3 replies

    The gap in this measurement is everything that moved to Discord, which for newer tools is most of it. Our forum yield would look terrible and it is because the questions are in a chat channel that is not searchable and will be gone in a year.

    That is worse for everyone. It just does not show up in a table like this.

    22
    Share
    Reply

    Answering anonymously, a moderator will review it first.

    Report
    • @measured_it_myself · 2w ago

      Agreed and it is the main limit of the method. I can only measure venues that publish. The honest version of my table is "of the forums that still exist, here is which ones carry demand", and Discord is a hole in it.

      16
      Share
      Reply

      Answering anonymously, a moderator will review it first.

      Report
    • @lurker_ratio · 2w ago

      Also worth saying a Discord answer helps one person. A forum answer helps everyone who searches it for six years.

      9
      Share
      Reply

      Answering anonymously, a moderator will review it first.

      Report
  • @lurker_ratio · 2w ago

    The Python line is doing a lot of work here and it is the only reason I believe the rest of the table.

    1
    Share
    Reply

    Answering anonymously, a moderator will review it first.

    Report
  • @support_lead_once · 2w ago

    799k views on one SSH key thread is a whole product decision hiding in a support ticket.

    1
    Share
    Reply

    Answering anonymously, a moderator will review it first.

    Report
  • @docs_are_the_forum · 2w ago

    What was the cutoff for question-shaped? That is where these methods usually leak.

    1
    Share
    Reply

    Answering anonymously, a moderator will review it first.

    Report