I’ve always been vexed by the question of how to validate a startup idea. While it may be the most well-worn topic in all of startup theory, there’s still no consensus on the best way to figure out if users actually want what you’re building. Despite cocksure advocates on both sides, there remain two incompatible schools of thought, and if you buy into the wrong one, you’ll waste a lot of time.

Authors such Steve Blank of Four Steps to the Epiphany and Bill Aulet of Disciplined Entrepreneurship say that if your idea is any good, you should first be able to get 5–10 customers to agree to buy it based on your pitch. You should of course ship and iterate on an MVP thereafter, but only if those meetings go well. If they don’t, then your idea stinks and you should fix it before you waste time shipping anything.

Advocates of The Lean Startup by Eric Ries disagree entirely. Ries says, “We could survey them [users] to get their opinion, but that would not be very accurate because most people have a hard time assessing their feelings objectively. Experiments are a better gauge.” Talk is cheap, and while you should certainly interview users, it should only be to understand their lives, not to ask whether or not they would use your product. Users are liars who don’t really know what they want, so the only way to get reliable feedback is to ship an MVP and observe how they behave.

Both approaches are well subscribed, but I haven’t heard anyone point out that they contradict each other. If users can’t accurately assess their own feelings, then interviews alone are misleading, but if they can, then it would be a waste of time to build an MVP without first observing measurable demand. So which is it?

Hollywood

It turns out there’s an industry that already understands this problem better than we do: Hollywood. Hollywood churns out movies the way Silicon Valley churns out technology products, and it’s well-accepted in Tinseltown that there in fact exists two types of movies, high-concept and low-concept, and the way you pitch your movie depends on which it is. High-concept movies can be “compressed” down to a one or two sentence description without losing too much in the process. These descriptions often take the form of a “what-if” scenario like “What if we cloned dinosaurs and put them in a theme park?” or “What if Batman and Superman fought each other?”, premises that can be easily pitched to production studios.

Low-concept movies, on the other hand, are based on complex plots or character development, so they’re difficult to summarize with any fidelity. Think Wes Anderson films or Star Wars. The writer can’t hook the producer on a premise alone, so the film must instead be sold on the talents and pedigree of the team. Star Wars was rejected by Disney, UA, and Universal, who called it “a little strange”, before eventually being picked up by Fox, who George Lucas would later say “invested in me, not the movie.”

Startups

Startups, it turns out, aren’t much different than movies in this regard. Some ideas can be compressed relatively losslessly down to a sentence or two, so they’re easy for users to grasp, while others will require higher-fidelity representations such as mocks or an MVP in order for users to really “get it”.

High-concept ideas are often pitched in the familiar “X for Y” format. My favorite example is Hotmail, the pitch for which was “Email on the web, so that you can access it anywhere”. That sentence says damn near everything you need to know about the product. It clearly conveys both the problem and the solution. The problem was understood by anyone who had ever wanted to access personal email at work, and the solution could be easily visualized by the user simply imagining their current email client, except now in the browser. In fact, the founders themselves were so aware of how much of the value was captured by the idea that they became quite cagey around VCs, fearing they might pass along their idea to competitors, as they explain in their Founders at Work interview:

The whole VC community has so many links with each other — you never know. Netscape was building email servers. What if the VCs were just to say to them, ‘Hey, why don’t you do web-based email?’ And that’s it, that’s the idea, right? There was not that much to protect in terms of IP. Whoever built it first would win the market.

But not all businesses are so easily compressed. Slack is perhaps the most notable recent example of a low-concept idea. Though one may be tempted to describe it as “IRC as a service”, this doesn’t capture what was unique about Slack, especially because Hipchat was already that. Slack killed Hipchat not with a silver bullet, but with a thousand lead ones. It had more reliable integrations, better uptime, a more forgiving pricing structure, and was all around better designed in hundreds of tiny ways. If you were to pitch Slack to Hipchat users, you would have no choice but to describe it as something like “A better Hipchat”, which hardly says anything at all. It’s no surprise, then, that Slack emerged organically from the needs of Stewart Butterfield’s previous company rather than running the typical gamut of user interviews.

As an aside, I should emphasize that, contrary to what some people say, high-concept is a property of the idea, not the pitch, which is an important distinction, since it means that it’s a fool’s errand to try and find a pitch that conveys a low-concept idea. Every idea needs an elevator pitch, of course, but realize that low-concept ideas will be unavoidably obfuscated in the process. “The Hero’s Journey in Space” may be an accurate description of Star Wars, but it’s an extremely lossy one.

So how do you know which type of idea you have? There’s a simple litmus test for a high-concept idea: does your elevator pitch clearly convey both the problem and the solution? It doesn’t even have to convince users that it’s a good idea. It may even be a bad idea. It just has to describe the product clearly enough so that users can decide whether or not it solves their problem, and it must do so in the span of an elevator pitch, not because you’ll only have 30 seconds to talk to them, but rather because that’s about all the detail you can hope to convey while still receiving reliable feedback. As you describe your product, users are simulating in their minds what it would be like to use it, and the more detail you throw at them, the more difficult this simulation becomes.

The upshot is this: The lower-concept your idea, the higher-fidelity your representation of it must be in order to receive reliable user feedback.

Heuristics

There is one complication, though. While it’s true that only high-concept ideas can be validated by just talking to users, not all can. It’s not enough for users to understand your value proposition. They also have to understand their own behavior. And, as Ries rightly pointed out, people can be quite bad at this. Even if someone completely understands your idea, they may still give you unreliable feedback, either in the form of a false positive or a false negative, since people are constantly and unknowingly operating on countless outdated heuristics about the world. One of the earliest lessons we learn as children, for instance, is not to get into cars with strangers, so even though the concept of ride sharing is easy to grasp, I doubt early conversations about Sidecar/Lyft/Uber were very promising. It was too scary.

A small number of intrepid, open-minded early adopters may risk violating conventional wisdom, but the rest of us only let go of old heuristics once we observe that some fraction of the population has deemed it safe to do so.

So how can you distinguish between reliable feedback and feedback based on outdated heuristics? The fun thing about startups is that this part is an art, not a science, but at the very least, if you’re going to interpret user feedback through this lens, you should have a theory for a) where the heuristic comes from, and b) why it’s now wrong.

Taking this into account, Here’s a simple algorithm you can use to validate your idea:

(high-concept) AND !(fighting outdated heuristics) → Congrats, you have a receptive audience who understands what you’re doing. Just ask them if they want it.

(high-concept) AND (fighting outdated heuristics) → People understand your idea, but most dismiss it out of hand. You should still validate by talking to users, but only to fearless early adopters.

(low-concept) → No one knows what the hell you’re talking about. Validate using high-fidelity mocks or an MVP.