I Cloned 2,000 Hacker News Users to Predict Viral Posts
Michael Taylor / Also True for Humans
Length12m
About this audiobook
Can AI predict what will go viral online? That's the question at the heart ofMichael Taylor’s latest experiment, in which nearly 2,000 AI personas based on real Hacker News commenters were tasked with predicting which headlines would take off. The resulting 60 percent accuracy rate was significantly better than chance, but with revealing limitations: The social dynamics that determine virality (those early upvotes that create momentum) introduce an element of chaos that AI models can't fully capture. Michael balances out his technical insights with practical takeaways for using AI in market research and a prompt template for you to try this approach yourself.—Kate LeeWas this newsletter forwarded to you?Sign upto get it in your inbox.I created 1,903 AI personas based on real Hacker News commenters and asked them to predict which headlines would go viral. They got it right 60 percent of the time—20 percent better than flipping a coin.That's a meaningful result.Chief marketing officers saythey'd use AI market research if it matched human responses just 70 percent of the time. At 60 percent accuracy, we're close enough to matter—but my experiment also revealed why it’ll be difficult to do much better.The excitement around my originalHacker News simulator postwas understandable: If AI could reliably predict viral headlines, you could keep testing until you found one that hits the jackpot. Using AI would be much faster and cheaper than traditional market research as well. But the 100-plus people who reached out after my post were skeptical that machines could match real focus groups. Marketing is a number-driven game, and marketers are finely tuned bullshit detectors. They wanted proof.So I ran the experiment. I pulled 1,147 headlines from a single day and asked my nearly 2,000 personas to pick winners from a mix of top stories and flops. The 60 percent accuracy rate was encouraging—but when I dug into which headlines the AI got wrong, I discovered something more important than the success rate itself. The problem isn't just predicting individual behavior. It's that viral content depends on social dynamics that compound in unpredictable ways. Even if I could perfectly model your choices, you're influenced by how many upvotes a headline already has when you see it. One extra early vote can change everything—sending identical content down completely different paths in parallel universes.Here's what I learned about the promise and limits of AI market research, and why achieving a useful-but-imperfect level of accuracy in predicting viral headlines might be the best we can do. I’ll also walk you through the prompt template you can use to try this yourself in ChatGPT or Claude.The experiment: Is ChatGPT an Oracle?Become apaid subscriber to Everyto unlock this piece and learn about:Why social dynamics make virality difficult to predictWhat responsible use of AI-powered market research looks likeHow to make your own AI commenter armyUpgrade to paidClick hereto read the full postWant the full text of all articles in RSS?Become a subscriber, orlearn more.
