When you’re building an entirely new kind of product, how do you measure success?
And is that the same for every kind of product, in every sector—nonprofit, government, tech, whatever? Alistair Croll, Eric Ries, and Danielle Morrill discussed these questions in depth, during an hour-long webcast on October 2nd. Here’s a recording of their conversation, and it covers some fresh ways to approach metrics.
You’ve told us clearly that you want more in-depth how-to sessions at The Lean Startup Conference. So this year, we’ve packed in more talks and workshops with detailed advice than ever before; check our program page for details on our initial batch of talks. Take advantage of these sessions by registering today for the best price possible; our fall sale ends on Friday, October 31, and it’s the last price break of the year.
To give you a taste of our how-to sessions, we asked three of our conference speakers for tips you can put into practice right now–whether you work at a startup or in an established organization.
1. How Best to Engage Customers Remotely
You understand the importance of engaging directly with your customers as you develop products for them, but what happens when your user base is very far away? “There are a ton of tools out there that allow you to communicate with someone remotely, and when choosing a tool there are a lot of things you need to keep in mind,” says user experience consultant Holly DeWolf, whose conference session will teach practical, cost-effective techniques for remote engagement. “Do you need to see your customer’s computer screen? Do you need to record the session to share it with your team? Is it user-friendly for you and your customer?”
When connecting with distant users, DeWolf recommends determining which tool is least invasive for your customer base. “The big thing for me is the usability of the tool itself for customers,” DeWolf says. “I try hard not to make a customer download a tool they don’t already have. Tools like Skype or Google Hangout are popular now, but not everyone has them downloaded already. [If they don’t have them], I prefer tools like Join.me or GoToMeeting that make it easier for a customer to just click on a link and get on quickly [without having to download anything].”
DeWolf also recommends that you practice using a tool in advance to work out any kinks. She explains: “Ask a coworker, a friend or even your grandma to get on so you can practice once or twice before getting on with a customer. Where are you going to keep a list of the questions you’re going to ask? Where are you going to take notes? Are you recording the session? Do you need a note to remind you to do that? There is going to be a lot going on, so as you start adding in little things, it makes it really worthwhile to have practiced beforehand.” In conducting our own customer interviews, we’ve found practice to be critically important for success–but easy to forget. Schedule it on your calendar to make sure it happens.
2. How to Interpret Customer Feedback
When you hold interviews with potential customers for your new product, it’s tempting to hear anything they say that isn’t overtly negative as a sign of enthusiasm. But be very careful in reading their tone. “If you’re talking to people, and they’re very polite and mild-mannered the whole time, that’s a sign that you’re not really solving a big problem,” said Cindy Alvarez, author of Lean Customer Development: Build Products Your Customers Will Buy and head of product design and user research for Yammer (a Microsoft company). In her session at the conference in December, Alvarez will lead entrepreneurs through live problem-solving to help you with the challenges of customer development and qualitative feedback.
“I’ve never seen an interview case where people who were enthusiastic customers did not express some sort of frustration or excitement,” Alvarez said. “If you don’t hear the variation, if you’re not putting exclamation points in your notes anywhere, then you’ve got a bunch of polite people who probably won’t buy your product. I’d say if you talk to five people and none of them seem particularly enthused, then try talking to a different type of five people.”
It’s deceptively easy to test the wrong aspect of your business or run the wrong test for what you need to learn, wasting time and money while you head down the wrong path. “Entrepreneurs always think they have the best idea. They do need that confidence to execute it and make it happen. But you have to balance that confidence with an awareness and open-mindedness that you could be wrong and don’t know until you test that idea with potential customers,” says Grace Ng, co-founder at QuickMVP. At the conference, she’ll lead a session, “Tactics for Truly Effective Experiment Design,” which includes a decision-making framework that guides teams to choose the right kind of experiment to run based on what you need to test.
When you create an experiment, Ng notes that constructing an accurate hypothesis to test should include clearly stating the problem you’re solving and who you’re solving it for. “A hypothesis statement is the building block of an effective experiment,” Ng says. “An accurate hypothesis will state, ‘I believe this solution of X will solve this problem of Y, and in order for this to be true, I need to see these results of Z happen’” If your hypothesis doesn’t prove true, you learn a lot and can adjust accordingly; if you run a test with no hypothesis (“let’s just see what happens”), it’s much harder to figure out what step to take after.
For a whole lot more relevant how-to advice, we’ve collected some of our favorite talks below from previous Lean Startup Conferences. And for more in-depth discussions on building your new products and services, join us at this year’s Lean Startup Conference. Register today. Our our fall sale ends Friday, October 31, and it’s the last price break of the year!
— Some of our favorite how-to talks from Lean Startup Conferences past —
A gratifying aspect of teaching people to use Lean Startup methods is seeing the ideas take root in more and more sectors. For a couple of years now, Lean Startup has been spreading in mission-driven organizations, where people commonly refer to the principles as “Lean Impact.” If you work in government, in an educational institution, at a place like Ideo.org or Kiva, or in a startup non-profit, we have a growing body of resources to meet your need for information about applying Lean Startup when profit isn’t the sole goal.
First off, we’re pleased to report that Leanne Pittsford—founder of Lean Impact, Start Somewhere, and Lesbians Who Tech—is leading a full-day workshop on Lean Impact essentials at this year’s Lean Startup Conference in December. She and several hand-picked guest speakers will explain how they’ve used Lean Startup principles to achieve greater social impact, and they’ll answer pressing questions about funding models for mission-driven organizations that need to experiment with new products and services. Our fall sale for conference tickets ends on October 31, and it’s the last price break of the year, so register today for discounted pricing to avoid paying full price. (We also offer scholarship passes for young and minimally funded non-profits; if that’s your org, apply right now.)
If you want a taste of what that workshop will cover, join us for a free webcast on October 28 at 10a PT: An Introduction to Lean Impact, in which I’ll talk with Leanne about how social-sector organizations can use the Lean Startup framework to meet their goals, and we’ll answer audience questions live.
In addition Leanne’s workshop, this year’s Lean Startup Conference features a number of other speakers from social-sector organizations:
Christie George, a socially-focused investor who recently wrote a great piece on funding alternatives, and Mitch Kapor, well known for his impact investing, will discuss measuring success beyond the bottom line
Max Ventilla will explain how AltSchool is building an ambitious new network of customer-driven schools
Julie Lorch from DoSomething.org will speak about building cross-functional teams
Tiffani Bell will will talk about learning quickly in order to address Detroit’s water crisis
Margo Wright will speak about the nuances of customer development when you’re working with a population you know well already.
Reverend Ken Howard will speak about creating and sustaining Christian communities using the Lean Startup approach.
Those examples draw from our initial group of speakers, and we’ll be announcing more later this month.
In the meantime, we’ve gathered below a slew of videos from previous Lean Startup Conferences that have great lessons from mission-driven organizations. Enjoy—and we look forward to seeing you at our webcast on October 28 and at the 2014 Lean Startup Conference in December.
In our “Now What?” series, startups ask real-life questions, and we find experienced entrepreneurs to offer deep, relevant advice. If you have a startup challenge, and you’d like insight from an experienced entrepreneur, let us know in this short form. – Eds
Minute Sitter’s issue
We’re in the process of testing the market for an iPhone app that helps parents connect with local, trusted babysitters and after-school care providers. Of the parents we’ve interviewed in our target market so far, this product resonates very strongly; most have rated the problem of not being able to find a babysitter a 7 or 8 out of 10. The problem is that it’s very difficult to find more interviewees. We’ve exhausted our friend network, and when we ask parents at the end of the interview if they know of others we could speak to, they always say yes, but it doesn’t progress any further. We’ve tried connecting with parenting Facebook groups to no avail, and it seems the next option is to pay for interviews, which we don’t want to do as this taints results. How do we find more interviewees?
About the expert, Cindy Alvarez
Cindy Alvarez is the author of Lean Customer Development: Build Products Your Customers Will Buy. She runs User Experience for Yammer (a Microsoft company) and has been helping companies build better products through intensely understanding their customers for over 14 years. Her background spans psychology, interaction design, product management, customer research, and lean startup tactics. She tweets and blogs.
About the reporter
April Joyner writes on business, entrepreneurship, and technology. She was previously a senior reporter at Inc., and she has also written for FastCompany.com, NewYorker.com, and OZY. She lives in Brooklyn, New York, and enjoys playing the violin in her spare time. Follow April on Twitter.
Interview with Cindy Alvarez, September 2014. Edited and condensed here.
April Joyner: What’s the first thing this startup should do?
Cindy Alvarez: The first thing would be to make sure that within their team, they’re aligned on what their hypothesis and target customers are. You need to boil things down to a sentence that is as simple as I believe this kind of person has this kind of problem, which could be solved in this way. And those have to be fairly specific—specific enough that you might think of a person that meets those criteria. A lot of times, I’ve found that teams aren’t actually in agreement on the hypothesis, even if they think they are. Once you have that, you basically have an implied audience in there. So then you start thinking about, “Who is that person, and where are they likely to be?”
Steve Blank has this great pyramid of needs. It’s basically a person who has a problem, recognizes they have a problem, has the ability to solve that problem, and has tried to put together a solution out of bit pieces. Those are the people you want to start with, because they’re the people who are the hungriest. This startup’s customer might be anyone who has a child, but that’s not necessarily your best market. That might be your eventual market, but the people you need to start with are the people who have met all those other criteria.
AJ: In this startup’s case, what might those criteria be?
CA: “Has a problem”: We define as people who have children who need care. “Who recognize they have a problem”: Something in their life is frustrating to them because they don’t have care. Maybe they’re not having date nights, maybe they’re not going to networking events, maybe they’re not seeing their family, or they’re not able to play a sport. You might have kids and not have a care problem. In that case, maybe your life would be happier if you had a babysitter and did this extra thing, but you’re not feeling it right now.
The next thing would be ability to make a change: In this case, probably someone who can pay. Your very lowest-income client is probably not the best place to start, because care is expensive in any situation. Then, beyond the ability to deal with it would be someone who has actually tried something—someone who has tried to find babysitters before, especially via some kind of online solution. If someone says, “Oh, yeah, I like to have a baby sitter,” but they have never made any attempt to acquire one, then they’re probably not the best customer for you.
Another thing that’s helpful is to do what I call a traits continuum, which is basically to write opposing traits, one on one side, one on the other, and figure out where you think people are. For example, for anything app-related, you might have from tech-savvy to not tech-savvy. Way over on the “not” side might be someone who’s never downloaded an app to their phone—probably not a good candidate here. A little further over might be someone whose significant other or kids put apps on their phone, but they don’t know how to do it. That’s still probably not the best market. Way at the other extreme is the kind of person who will try anything and download anything.
AJ: So once you know who your target customer is, what do you do next?
CA: A lot of times, if you have this list of traits, then you can say, “Oh, this person will be a great person to talk to.” You don’t necessarily need to identify every individual person you’ll speak to. But if you can read your list and it isn’t specific enough to make you think of even one unique person, you might be starting too wide. If you actually manage to tie it back to that one example person—like, “Oh, this sounds just like Pamela”—go talk to Pamela and say, “Do you use Craigslist? Do you use Sittercity? Do you use Care? How do you get babysitters today? What parenting groups do you subscribe to? Are there mailing lists? Are there children’s activity places that you go to?” Asking someone that level of information isn’t a strict customer development interview, but it’s saying, “Okay, we’ve identified a persona. Now let’s actually find that person and ask them where we should start looking.”
From there, try and figure out how can you convince that person to talk to you. In the software world, a lot of these pitches for interviews happen online. So, it might be constructing your email pitch. I typically recommend that people test it out first. Don’t blast out your email pitch as soon as you have it written. Send it to one or two people, maybe not even people who are your target customers, and say, “Read this, how does this sound?” A lot of times, the first draft email will not have the tone that you intend. A friend of yours might say, “This sounds arrogant,” or “This sounds too informal,” or “This sounds overly formal,” or, “This sounds like I’m not sure what you’re going to ask of me.” Other people are very good at picking up on those little weird bits of language that influence response rate. So I might write a pitch, send it to someone, get their feedback, change the language to take care of any issues, and then start sending it out to other people who I think are my real target customers.
AJ: To backtrack, since you mentioned email: when you’re brainstorming where customers might be, you’re actually gathering emails?
CA: It depends. There are some places where you might be able to contact people somewhat directly through the site, like LinkedIn or Quora, for example. That’s probably a less good option for a parenting-specific site. For that, the places where you find people might be, say, Parenting Mailing List. A mailing list is a place where you’d be able to get someone’s email address fairly easily, but something like a parenting forum is not so much. Generally, posting to a forum to say, “Hey, do people want to talk to me about my business idea?” is seen as sort of a negative thing. That’s along the same lines as advertising your product, and no one really wants that. In those cases, you may need to become a contributing member of that community first, invest that time until you get to know people. At that point, you have a little more social acceptability to ask questions. Or you can make friends with individuals and get their contact information that way.
With parents, the real world is a very, very good place to find them. Let’s say you have friends who have friends who are parents. Then what you are probably going to do is ask your friend to forward them an email. But your friend isn’t necessarily going to want to do that unless you’ve made it very clear that you’re going to be a good actor. I wouldn’t ask you to introduce me to a friend, and then say, “Go ask your friend to help me move my house.” What’s typically easy is to send an email to someone saying, “Hi, I’d really like to talk to parents in order to learn blah, blah, blah. You have a friend who I’d be particularly interested in talking to. Would you be willing to forward this email to them to make the request? I promise it will be no more than a 20-minute conversation, and if they don’t respond, I won’t continue bothering them.” There, you’ve established very clear parameters of what you’re asking for.
“TO BE HONEST, WHAT THIS SMACKS OF TO ME IS NOT THAT THEY’VE RUN OUT OF PEOPLE TO TALK TO, BUT THAT THERE’S SOME OTHER PROBLEM.”
AJ: So, with this startup, it sounds like they’ve talked to some people, but they’ve exhausted their network. In that particular instance, how can they find other people?
CA: To be honest, what this smacks of to me is not that they’ve run out of people to talk to, but that there’s some other problem. Personally, as a parent, I think it would be pretty easy to get me to agree to an interview about this topic, because it’s a pretty painful thing when parents have a hard time getting care. The fact that people aren’t champing at the bit suggests to me that either the problem isn’t being pitched in a way that’s resonating, or that the request for conversation sounds onerous in some way. Maybe it sounds like it’s going to take a really long time, or they might be trying to contact people via phone who would rather use email, or vice versa. They might be trying to call people during the dinner hour. If you try to get anything out of me between 6:30 and 8:30 pm, I am not receptive.
What’s tricky is that people usually will not tell you what the problem is, so you have to do a certain amount of troubleshooting. If someone was like, “Hey, my friend really wants to talk to you about this,” and I kept saying, “Oh, yeah, I keep meaning to talk to her,” that’s a sign that for some reason, it’s not valuable enough to me. So the founders need to take a look and make some guesses about what that problem is. Is it that their pitch is not compelling enough? Are they communicating with people in a way that’s somehow off-putting, but they may not realize it? Are they communicating at people at the wrong time? Are they communicating with people in achannel that’s not common to them? Are they coming off somehow as advertisers? There are lots of things like this that might actually be an issue.
AJ: OK, so where should you account for these things? Is this something you need to think about right when you define who your target customers are?
CA: Whenever you’re going to interview customers, there’s a few things that you need to know about, what they value and what their limitations are. Typically, that’s what your network is for. You don’t necessarily have to be deeply embedded within your customer market, but you should at least be within arm’s reach of them. In this case, surely they have parents who are friends, or they have friends who have parents, who are friends. If they don’t actually have, say, five or six people that they can have a simple conversation with along the lines of, “What is the best way to reach you? Are there certain times that are a dead zone?” I think that’s a very difficult place from which to start a business, and I would recommend that they start making some friends who fall into that category.
You’re obviously never going to predict all of these things. But what you can do is constantlyiterate on the process. People who aren’t willing to talk to you may still be willing to answer a single question via email. If you say, “I’d love to talk to you about this solution,” and someone doesn’t respond, instead of continuing to try, you might say something like, “I’m just curious. We don’t have to have this conversation, but did I ask in a way that was inconvenient to you?” Or “Is there a way I could have phrased this better?” Or “Is there something I could have done to make this seem more appealing?”
That allows someone to give you a little bit of honesty without having to commit to a 20-minute phone conversation. You’re not going to get a whole lot out of one question that will stand in for a customer development interview. What you want to do is make sure that the next time you contact a parent, she actually says, “Yes, I want to talk to you.”
“YOU’VE JUST SPENT A PARAGRAPH TELLING ME ALL ABOUT YOU. I KNOW PEOPLE DO THIS TO GAIN CREDIBILITY. IT SEEMS LIKE A VERY LOGICAL STRATEGY, AND YET IT FALLS FLAT ON ITS FACE.”
AJ: So this goes back to what you said earlier about developing a strategy to convince people to talk to you. Are there certain things that make people more or less likely to want to talk?
CA: Sure. You want people to talk to you; you want to recognize them as experts.
There are a few things that I can list that are turnoffs. One is when people feel like you’re trying to sell them something. You want to be really clear that you’re not doing that. At Yammer, I sometimes will start conversations with prospective customers by saying, “I’m not a salesperson. I couldn’t sell you this product even if I tried.” Eventually, of course, you’re going to ask people for their money, but when you’re doing the customer development interview, you want to remove that from the conversation. In fact, I don’t even like using the word customer with prospective customers, because I don’t want them in that buying mindset. So I’ll use words like you or your personal experience or in your life. I won’t say, “You seem like a prospective customer,” or “You might be a future customer,” because then I think they’re in that mindset of “At some point this person’s going to ask me for money.”
The second one is ego. People will say, “I’m a marketer with 20 years experience and blah, blah, blah.” If that’s the start of your pitch, then your email is basically saying, “Here’s a bunch of stuff about me.” So I’m not really convinced that you want my opinion because you’ve just spent a paragraph telling me all about you. I know people do this to gain credibility. It seems like a very logical strategy, and yet it falls flat on its face. I’ve gotten unsolicited customer development interviews where people go on and on about their credentials. I’m just like, “Ugh.” I barely read on.
I’d say the third one is an unclear ask for your commitment of time. Sometimes I will get a pitch and someone clearly wants to learn something from me about a product, but I don’t know what they want from me. Someone reading your email pitch or hearing your verbal pitch should have a very clear sense of what you’re asking for. I think people try to be polite—we think, “I won’t come right out and ask for things because that seems rude.” But giving multiple options is actually more of a burden because now I have different decision points to consider. The best pitches are very straightforward: “Can I talk to you for 15 minutes on the phone?” That is incredibly clear. I know exactly what I’m committing to, I know the medium, and I know it’s not going to take that long. I’m very likely to say yes to a pitch like that.
I think it’s generally best for you to pick a modality that you like, and offer another one as a fallback. Someone might say, “Look, I’d love to talk to you but it’s really hard to get me on the phone.” Then, you can say, “Can we converse via email or via chat instead?” At the last couple of companies I’ve worked at, I’ve had a large number of international customers. Between bad phone connections and accents on either side, either me not understanding them or vice versa, sometimes people will say, “Let’s just do Gchat.”
“I’LL SEND IT TO MYSELF AND OPEN IT ON MY PHONE. AND I’LL BE LIKE, ‘OH, THE ASK IS WAY BELOW THE FOLD. THIS IS TERRIBLE.’”
AJ: When you’re asking people to do customer interviews, has there ever been anything that’s surprised you about the process?
CA: Well, I don’t know if it’s surprising, but I think something that catches me off guard is mobile. At this point, more than 60% of emails are opened first on a mobile device. That is an incredibly short amount of space in which to make your point. If I’m looking at something on my iPhone screen, I’m seeing maybe two sentences. Somewhere in that two sentences, you have to hook me, and it has to be really clear how with one thumb I can hit reply and say yes. If not, it goes into the read-but-not-replied-to depths of my inbox. That’s purgatory. So I’ll write what I think is a really good succinct pitch, and I’ll send it to myself and open it on my phone. And I’ll be like, “Oh, the ask is way below the fold. This is terrible.” So I have to go back and cut more words.
The other thing is that people who are less tech-savvy have a very itchy spam filter. In talking to a lot of Yammer’s customers about exploring new features, a lot of the folks who are outside of technology are very suspicious that things might be some kind of spam or phishing. A lot of times, we’ll send one email to someone and see if it gets picked up on, and then send a few more, versus trying to blast people all at once. I’ve been very surprised sometimes by people who write back saying, “Are you a real person?” And I’ll read the email, and I’m like, “I don’t know what they’re responding to.” It seems completely legitimate—it’s from a real person, I’ve written it in a very human tone of voice, but something tripped someone’s “Maybe this is a phishing attack” filter.
I think one big thing is sending an email from an account that’s not a real name. People are suspicious of things that don’t come from humans. If you send from “MinuteSitter Support,” that’s not a human. If the email says it was sent from “Cindy, MinuteSitter,” that’s slightly better, but that might be someone selling me something. Another thing is, if you’re using a service like MailChimp or CampaignMonitor, sometimes the way the “sent from” line is rendered looks suspicious, like if it says, “From X on behalf of Y.” I’ve found that when things seem like they’ve been emailed through an additional domain, non-tech-savvy people don’t understand what that means—they just think it’s probably bad. At the startup level, I would just send emails from my personal account.
AJ: How can you tell whether you’re not asking people for interviews the right way, or if your product just doesn’t resonate with people in your target market?
CA: The easiest thing is to find some other person, even outside the target market, pitch them, and ask for feedback. So if the startup has an email drafted, forward that email to someone completely outside of their organization and say, “What do you think about this email? Would you be likely to say yes? What do you think about the people who wrote it?” It’s so valuable to have a friend outside the building for things like this. I just have a couple of friends, or people I’ve worked with in the past—at any given time I might send them an email and then follow up via chat, and be like, “Did you get that email? Was there anything weird about it?” Just the ability to do that saves so much time. If you don’t have that person, find that person in the startup community.
This is also something you could do with a quick survey. We’ve done this for feature work at Yammer, when we’re trying to ascertain whether the tone of our copy is positive for people. We might show a screenshot that has a bunch of copy on it, and then on the next page of the survey just ask a couple of questions like, “What did you think this was asking for? What did you think was happening in this step?” and see how people respond. We’ve definitely had cases where certain words had a certain connotation that people were picking up on. So we might show a screenshot, and then on the next page people would say, “Oh, I thought this was going on because this term seemed very negative to me,” and it’s often very surprising.
If they actually got people to respond to their pitch, and no one identified any issues with it, then I would move on to the next easiest thing to validate, which is, “Does this solution make any sense to their target audience?”
AJ: How do you validate that?
CA: Once people have agreed to talk to you, you want to know what they’re doing today. One trap startups tend to fall into is to ask aspirational questions. It’s typical to say something like, “Would you be interested in a service that does X?” That’s an almost useless question. The odds are that you’re going to get a “yes” answer, because frankly, it’s free to say yes. There’s no commitment involved. If you say, “Would you like a service that delivers chocolate to your house every night?” I’d say, “Sure.” Never mind that I’d have to pay for it, or that my health might suffer. The other thing is that most people have things that they wish they would do. If you’re asking about future behavior—”If you had this service, would you do X?”—people are just terrible predictors. It’s not just that they’re likely to say yes; they’re likely to be wrong.
Instead of asking, “Would you like to use a service like this?” you want to take it a step back and say, “Tell me about how you have found care for your children in the past.” Then you’re going to get answers like, “I’ve used this online service,” or “I’ve never used an online service,” or “I asked the person who lives next door because I know them, because I’ve lived next door to them for ten years,” or “I asked my friend who already has a babysitter how she found hers.” These are going to be useful bits of information, and you’re going to use them as a jumping-off point to figure out how, from that past behavior, you can shunt people into a new behavior.
A lot of times, by talking to people about what they’re currently doing, you can uncover their frustrations with what they’re currently doing. For example, someone might say, “Oh, I don’t have a problem getting a babysitter. I just ask my friend Joyce, who has a babysitter that she really trusts, and I just ask Joyce for that babysitter’s number.” The frustration might be that sometimes the babysitter’s already committed to Joyce, or frankly, Joyce is getting annoyed that you’re poaching her babysitter, and you don’t want to lose a friend over it, or that this babysitter’s great, but she doesn’t drive. Those little bits of frustration are where you can identify opportunities for providing a better solution.
“IF YOU’RE TALKING TO PEOPLE, AND THEY’RE VERY POLITE AND MILD-MANNERED THE WHOLE TIME, THAT’S A SIGN THAT YOU’RE NOT REALLY SOLVING A BIG PROBLEM.”
AJ: How do you know you’re moving in the right direction, once you’ve tweaked your pitch and started talking to people?
CA: If you can get people to talk to you, you’re moving in the right direction. And once people have started talking to you, you should be listening for emotion. If you’re talking to people, and they’re very polite and mild-mannered the whole time, that’s a sign that you’re not really solving a big problem. I’ve never seen an interview case where people who were enthusiastic customers did not express some sort of frustration or excitement. Another big one is shame—people who feel like they ought to be doing something but they aren’t.
If you don’t hear the variation, if you’re not putting exclamation points in your notes anywhere, then you’ve got a bunch of polite people who probably won’t buy your product. I’d say if you talk to five people and none of them seem particularly enthused, then try talking to a different type of five people. It’s very unlikely that you’re going to strike out five times in a row, if you’ve really got a good market pitch.
AJ: Is there a certain number of interviews you need in order to figure out whether your product is resonating with people?
CA: The number of interviews people do is going to vary. A lot of times, I’ve said anywhere from 30 to 50 for this kind of scenario, where someone is just getting started. That person may be on the brink of making a big decision like, “I’m going to quit my day job,” or “We’re going to hire a full-time engineer,” or “We’re going to raise money.” Those are giant decisions. So 30 to 50 interviews are a lot, but if you’re deciding to quit your cushy day job and jump full feet into something, a lot of people want to have a sense of comfort. If you’ve already started a company and made those big decisions, then to some degree, you’ve already taken on that risk, so you might do fewer.
If you are able to very rapidly put out a minimum viable product and get people using it, then again, you might do fewer interviews because you’re going to be actually building the solution. Certainly, I’ve known people who’ve done five to ten interviews, but within the next week, they were able to put out a minimum viable product and get real customers using it. So they say, “Well, we are going to do a few interviews because now we’re actually watching people use the product, and people are giving us money,” which of course is the strongest possible signal.
If you have a startup challenge, and you’d like insight from an experienced entrepreneur, let us know in this short form. – Eds