AI Idea Validator Comparison: Testing the Leading Tools

If you've searched for startup advice recently, you've probably come across AI idea validators.

There are dozens of them now. Type in your idea, wait a minute or two, and you'll get a score out of 100 along with a recommendation about whether you should build it.

Out of curiosity, I decided to test a number of them using exactly the same business idea.

Rather than using a genuine startup idea, I deliberately chose Pets.com.

It sounds odd to use a company that failed over twenty years ago, but that's exactly why I chose it.

Why Pets.com?

On the surface, it looks like an excellent business. People own pets. They buy pet food every month. Nobody enjoys carrying 20kg bags home from the supermarket. The market is huge and continues to grow.

If you fed that description into most AI systems with no historical context, you'd probably expect a fairly positive result.

The problem is that we already know how this story ended.

Pets.com became one of the most famous failures of the dot-com era. It burned through huge amounts of investment before collapsing less than a year after its IPO. It wasn't because there wasn't a market.

It was because the economics of the business were much harder than they first appeared. Heavy products. Thin margins. Expensive delivery. High customer acquisition costs. Customers who often just bought pet food with the rest of their weekly shopping anyway.

That's what makes it such an interesting test case.

The Results

Here's what the different AI validators came back with. Tool Score DimeADozen 97/100 ValidatorAI 78/100 IdeaProof 73/100 Foundra 62/100

They were all surprisingly positive about what is famously a bad business idea. The reason they gave these scores was because they all broadly said the same thing: The market is large. Consumers value convenience. Find a niche. Build an MVP.

None of that is wrong, but I don't think it's answering the question founders actually care about.

The Market Was Never The Problem

This is the thing that struck me most: almost every tool spent a lot of time explaining why people would like pet food delivered.

Of course they would. I would.

The problem is that liking something isn't the same as building a viable business around it.

People already buy pet food from Amazon, or their supermarket, or Chewy, or Pets at Home. Quite often they add it to an order they were making anyway.

The question isn't whether people want convenience; the question is whether they'll change their behaviour enough to support another company doing exactly the same thing.

That's a much harder question.

What I Think These Tools Miss

Most idea validators seem to work by looking at a number of common dimensions. Market size. Competition. Timing. Differentiation. Execution.

They're all sensible things to look at, but the problem is that they don't always interact with each other. A huge market can still be a terrible business, and an industry with enormous demand can still destroy companies.

History is full of examples.

If AI mostly rewards large markets and obvious customer pain points, then I worry it ends up encouraging founders towards exactly the same kinds of businesses.

Building SignalCraft Changed My Thinking

While I was doing this experiment, I also ran the same idea through SignalCraft. One of the things I realised while building it is that I don't actually think founders need another idea validator. There are already plenty of those.

Instead, I think founders need something closer to market intelligence. There's an important difference.

An idea validator tries to answer: Is this a good idea?

SignalCraft tries to answer: What does the available evidence actually support?

They're not the same question.

In the Pets.com example, the conclusion wasn't "don't build this."

It was closer to: "The evidence suggests there may be opportunities in specialist niches or local delivery, but there isn't much evidence supporting another generic national pet food delivery business."

More importantly, it also highlighted the evidence it couldn't find. Things like willingness to pay. Retention. Price elasticity.

Those aren't things an AI should pretend to know.

Maybe We're Asking AI The Wrong Question

The more I looked at these reports, the more I wondered whether we're expecting AI to do something it can't really do.

It can't predict whether a business will succeed. Nobody can. What it can do is help founders understand the market they're entering. It can surface customer frustrations. Show what competitors are doing. Highlight recurring complaints. Identify assumptions that still need testing.

To me, that's far more useful than being told my idea scores 81 out of 100.

Final Thoughts

I don't think AI idea validators are useless.

Far from it. They're good at quickly summarising a market and getting founders thinking. But I also think they're becoming a crowded category, and many of them end up giving remarkably similar advice.

Perhaps that's inevitable. If every AI is trained to identify the same patterns, then every founder risks being nudged towards the same ideas. AI models are designed to give the most statistically likely answer, which from point of view of the user is the most average answer. People building a business seldom succeed by being average

Good founders don't win because they build what everyone else is building. They win because they spot something everyone else has missed. That's the kind of thinking I think AI should be helping with.

What to use instead of a validator score

A tool comparison is more useful when you know what good validation should actually contain. The evidence-based business idea validation guide sets out that benchmark, while AI Idea Validators: What They Can and Can’t Tell You explains the limitations behind the scores.