AI idea validators are not very good at validating ideas
If you’ve spent any time in startup communities recently, you’ve probably seen the explosion of AI business idea validators.
Describe your idea in a text box, wait a minute or two, and receive a polished report telling you whether it’s worth building.
On the surface, it sounds like exactly what every founder needs.
The problem is that many of these tools are trying to answer a question that isn’t actually answerable.
Validation isn’t prediction Most AI validators work in roughly the same way.
They search online discussions, look for competitors, identify pain points, summarise trends, and then generate a recommendation. If lots of people are asking for something similar, that’s usually considered a positive signal.
On the other hand, if nobody is talking about it, the report often concludes that demand is weak or uncertain.
That seems logical.
Until you remember how many successful products nobody asked for.
People don’t always know what they want Steve Jobs famously said:
“People don’t know what they want until you show it to them.”
Whether you agree with every decision Jobs made, history has repeatedly shown this idea has merit.
Very few consumers were asking for a smartphone before the iPhone.
When the iPod launched, many commentators criticised it for lacking features that competing MP3 players had, including a built-in radio.
No-one was demanding a bagless vacuum cleaner before Dyson.
Looking only at direct demand signals, none of those products would have appeared obvious winners.
Innovation often creates demand rather than responding to it.
The danger of searching for exact matches Many validators implicitly assume that if enough people are posting online saying, “I wish someone would build X,” then building X is a good idea.
Sometimes that’s true.
Often it isn’t. People frequently describe solutions rather than problems. If users complain that spreadsheets are difficult, they may not actually want another spreadsheet. If they ask for “an app that does everything,” that’s probably not the solution either.
The real opportunity usually sits underneath the request.
Uber’s success was based on identifying the real pain point. People didn’t demand to be able to book a taxi via an app. They just wanted to know how long they needed to wait until their taxi arrived to collect them. No more stressing about how long they had to wait, no calling the office to be told ‘its just around the corner’ every time. Just being able to see it on a map.
Experienced founders spend far more time understanding the underlying frustration than counting how many people asked for a specific feature.
Validation is not voting One of the biggest misconceptions in startups is treating validation like an election.
Five hundred people asking for something doesn’t automatically make it a good business. Equally, five people discussing an obscure frustration doesn’t mean it’s insignificant.
Some of the best businesses solve problems experienced by relatively small groups of people who care deeply enough to pay. Meanwhile, millions of people can complain about something they’ll never spend money to fix.
Demand isn’t measured by volume alone. It is measured by the combination of pain, willingness to change, willingness to pay, competitive alternatives, and whether your solution genuinely improves their lives.
Those are much harder questions.
AI is excellent at summarising. It’s much less reliable at concluding. Large language models (LLMs) are remarkably good at organising information. They can identify themes, summarise discussions and present evidence clearly.
Where things become more difficult is the final leap from evidence to recommendation.
Two founders can look at exactly the same information and reach completely different conclusions. One sees an overcrowded market. The other sees proof that customers already understand the problem.
Download the Medium app Neither interpretation is objectively wrong. This is why reports that confidently conclude “Build this” or “Don’t build this” should be treated with caution.
The confidence often exceeds the evidence.
The real job of validation The goal isn’t to find permission to build. Nor is it to eliminate every uncertainty.
The goal is to reduce avoidable risk.
Good research should answer questions like:
What problems do people consistently experience? How are they solving them today? Where are current solutions disappointing users? Which assumptions about my idea still haven’t been tested? Where is the evidence conflicting or incomplete? Those questions don’t produce a simple score. They produce better judgement.
Founders still matter There’s a growing temptation to outsource thinking to AI.
Ask a chatbot for a business idea.
Ask another AI to validate it.
Ask a third AI to generate the business plan.
Before long, the founder becomes little more than the person pressing Enter.
The irony is that the most valuable part of entrepreneurship has never been writing the business plan. It’s making decisions under uncertainty.
That’s still a fundamentally human skill.
Large language models are trained on patterns in existing human knowledge. As a result, they’re exceptionally good at recognising established approaches and summarising conventional wisdom. They are naturally less equipped to judge genuinely novel ideas precisely because there is less historical evidence for them to draw upon.
AI is very good at telling you what’s typical. Entrepreneurship is about recognising when typical isn’t good enough.
AI should make you better informed. It shouldn’t replace your judgement.
Better questions lead to better businesses The most useful startup research doesn’t tell you what to build. It helps you understand the landscape you’re entering. It reveals hidden assumptions. It uncovers evidence you hadn’t considered. It highlights uncertainty instead of pretending certainty exists.
The founders who consistently build successful companies aren’t necessarily the ones with the best ideas. They’re the ones who interpret imperfect information better than everyone else. And that’s something no AI validator can do for you.
This is exactly the philosophy that led me to create SignalCraft.
SignalCraft doesn’t try to replace a founder’s judgement with an AI’s opinion. It doesn’t assume that the absence of direct demand means an idea is bad. Equally, it doesn’t assume that lots of people asking for something automatically makes it a good business.
Instead, it approaches the problem as a piece of commercial due diligence.
It searches for evidence across the market, identifies recurring themes, surfaces customer frustrations, analyses competitors, highlights areas of uncertainty, and assesses how strong the available evidence actually is.
Just as importantly, it makes a distinction between evidence and interpretation.
If people are discussing a problem but not your proposed solution, that’s evidence. Whether your solution is the right answer is still a judgement that only you, as the founder, can make.
Likewise, if very little evidence exists, that doesn’t necessarily mean the opportunity isn’t real. It may simply mean you’re entering genuinely new territory, where uncertainty is naturally higher. SignalCraft is designed to make those uncertainties visible rather than pretending they don’t exist.
SignalCraft deliberately stops short of saying “this idea will succeed” or “this idea will fail.” Instead, it assembles and analyses the available evidence, identifies where that evidence is strong, where it’s weak, and where uncertainty remains. That’s because no AI — not even one with access to the entire internet — can reliably predict whether a genuinely novel product will succeed.
Entrepreneurship has never been about following instructions. It’s about making better decisions than everyone else with imperfect information.
The role of technology isn’t to think for founders. It’s to help founders think better.
That’s the difference between an AI idea validator and founder intelligence.