Author Archives: Lean Startup Co.

End the Year with Our Favorite Talks of 2015

We love hearing about all the different ways that people are internalizing Lean Startup principles. Eric Ries’ scientific approach continues to embolden a global community of thought leaders, many of whom we heard from at this year’s Lean Startup Conference.

Before we give 2015 a rest, we’re looking back at the powerful talks that our conference attendees told us they loved most. So regardless of whether you were able to make our big event, we have eight important and entertaining bits of workplace wisdom to share.

1. The Modest Experiments That Became Huge Companies

Know this: Successful companies start by focusing on small target communities and growing their MVP from there.

Laura Klein’s (Users Know) lightning tips talk spotlights the small-scale experiments that today’s big tech darlings crafted on their way to global domination. Examples include focusing on problems affecting a core group of users (Facebook), building an internal tool whose effectiveness could be calibrated in real time (Slack), testing assumptions locally before scaling those ideas globally (Airbnb), and creating a product manually before getting into automation (Product Hunt).

2. The 10 Keys to Building a Strong Community

Know this: Sending your users personal emails, memorable swag, and other messages that they’re important keeps them involved with your brand.

Speaking of Product Hunt, founder Ryan Hoover knows how to stoke and support a rabid fanbase, and he has great advice for growing a community of your own. Highlights include seeding that initial base and personally welcoming new users, recognizing both good behaviors and the bad actors, and sharing mockups of redesign concepts with your users. “You’re changing their home,” he says of the importance of design transparency, “so you want to make them feel like they’re part of the design process.”

3. How to Intentionally Design a Lean Startup Culture

Know this: Map out the right kind of culture for your business with this cool new tool.

Strategyzer’s Alexander Osterwalder understands that the belief systems, attitudes, and values held by employees directly affect the products a company creates. He’s part of the team behind The Culture Map, which helps organizations intentionally shape the purveying ethos in an office. He shows us how to use this tool to design a Lean Startup culture.

4. Testing the Riskiest Assumptions First

Know this: Build stronger startups by testing the assumptions that could topple your business before you tackle the easy stuff.

Gagan Biyani has been involved with renowned startups Udemy, Lyft, and Sprig, companies that maintain their singular missions while changing what they’ve created over time. He shares the specific Lean Startup practices that’ve made these organizations so successful, including strategizing creative ways to solve customer problems and untethering your ego from the products your company creates.

5. Applying Lean Startup to Storytelling

Know this: Savvy movie producers and fiction publishers alike are crafting tighter audience feedback loops that allow them to edit their way to big hits.

There’s nothing Lean Startup about the typical Hollywood blockbuster. As Telepathic’s Prerna Gupta explains, studios generally sink hundreds of millions of dollars into a production and hope they have a hit on their hands. But there are ways to use data and audience feedback to create more compelling movies — and books — while keeping the artistic process intact. Gupta explains how to fine-tune both our storytelling methods and the business models supporting them, through examples ranging from The Martian to her company’s innovative text message YA novels.

6. Spreading Intrapreneurship Across an Organization

Know this: Instigate genuine innovation in your organization by creating a model for intrepreneurship that empowers people in all levels of the company to build and test new ideas.

So you’ve become your company’s innovation evangelist — how do you go from being a singular force to incentivizing widespread change? Intuit’s Bennett Blank breaks down the steps through which Lean Startup practices moved through his company, such as allowing coworkers to make these practices their own and giving people a safe space in which to test out new ideas.

7. The Playbook for Creating Product Market Fit

Know this: Make sure the products your business creates are the ones your target customers really want by taking a page or two from the Lean Product Playbook.

Dan Olsen, author of The Lean Product Playbook, takes us through the methodology behind his book, which includes determining the underserved needs of your specific customer base and creating a strong value proposition and MVP.

8. Lean Startup, The Musical

Know this: Definitely the funniest interpretation of Lean Startup ideas that we’ve ever seen. Really, you’re just gonna have to watch this one. Especially if you’re a TLC fan.

We’d yet to get serenaded by stories of innovation until we met designer William McDonnell.

What’s Next in 2016

On that (ahem) note, there’s more cool stuff on the horizon for 2016: we’ll host our annual flagship conference in San Francisco, of course. Save the date for the Lean Startup Conference Week on November 1 – 7. It’s the one week in the year where you’ll strengthen your foundation of the Lean Startup methodology, learn advanced strategies and tactics for implementing Lean within your organization, and meet experts from your industry who are facing similar challenges. Early bird registration opens in January.

And if you can’t wait til Fall to see us, there’s our inaugural Lean Startup Labs series, which are industry-focused summits around the country. We’re starting with an enterprise lab in New York City in February; later in the year we’ll have summits in Detroit and New Orleans on startups and social good, respectively.


By Lisa Regan   @mybldanube


Established companies are established. Which means they’re big, operate with layers of complexity, and have a set way of doing business.

And as these established—we call them enterprise—companies look toward new ways of working, and new products to create, it often happens that their company culture isn’t quite ready to support the transformation. Sure, leadership might be looking to make a change, but they get pushback from the product or business teams. Or self-starters on individual teams may be eager to experiment, but they get the sense leadership isn’t interested. To find out what works to shake up a big, entrenched culture, we spoke to Brant Cooper.

Brant is co-founder of Moves the Needle and author of The Lean Entrepreneur—in addition to his many years at startups—and helps large enterprise companies like Intuit, Capital One, and Hewlett-Packard shift traditional business practices toward innovative ones.

We also recently dove in with Brant to explore the role of leaders in continuous innovation in this webcast conversation with Carie Davis, Coca-Cola’s Global Director of Innovation and Entrepreneurship.

Lisa Regan: What are the obstacles to innovation in large corporations?

Brant Cooper: We actually hear the same story pretty often: A company has made a first foray into integrating Lean Startup practices, but they can’t move the new methods past a small product team or innovation group. That’s pretty much always because other parts of the business—legal, or regulatory compliance, or marketing—are worried about the risk to the brand. We start to hear, “What will our customers think about that? It might adversely affect our brand.” Or maybe there are legal and regulatory compliance considerations around accepting payments. And those are all valid concerns. Big companies have to protect their core business—it’s where they make all of their money. So while this idea of bringing startup practices to large enterprises is not really new, often it hasn’t worked because those companies are right when they say, “Listen, we can’t just become a startup.”

So then the question is: What can you do about that? And I think the first thing you have to do is: Ask. Talk to people. A lot of these innovation teams will just sit inside their work area, and they’ll go, “Oh, Legal will never allow us to do that.” And they don’t even ask! Pick up the phone, or go out for a cup of coffee, and just see what the response is. Like, maybe they’re worried that you’ll want to run an MVP that’s not fully baked, but they don’t know that an MVP is usually not going to be released to millions of people anyway—it’s for targeted early adopters. And once Legal finds out that you’re only interacting with a small handful of customers, they’re like, “Ugh, don’t bother me with this trivial stuff.”

But it’s also possible to actually get those operational teams—branding, sales, IT, legal, compliance, even HR—truly on board with new methods for creating products. In our training, we encourage people from those teams to join the innovation teams and the internal product teams so that they are able to experience first-hand what it’s like to run experiments. Then, they can understand why the teams are acting the way they are, and what these experiments mean, and what’s really happening when they’re interacting with customers. When I say “join,” I don’t mean that they’re sitting on the side and taking notes—they’re active participants. We ask them to provide ideas.

Another strategy we use is to train the business and operational teams to themselves act like an internal startup, where the employees are their customers. We ask them to think, “What are some products or services that we can roll out to our internal employees that will enable them to innovate better?” If legal or HR start re-framing their employees’ concerns as problems that they might be able to solve, then they can act like an internal Lean Startup and propose solutions. They can go and do customer development on their own employees, and then run experiments with employees to see which one of the ideas might actually work.


LR: At enterprise corporations, what’s the role of managers in guiding teams through entrepreneurial processes?

BC: This is a big question. How do you mentor internal innovation teams? It’s not the same thing as managing. A lot of managers have a lot of experience, and they’re very successful within large enterprises. But their skill set really is around managing known processes, versus mentoring teams to try to discover new processes. We’re finding that to be a very tough nut to crack. We do some “train the trainers” work, so that they can then do the workshops themselves and teach. But even beyond training the trainers, it’s: How do you teach leaders to mentor the teams, not just manage them?

Here’s an example: In one of the sessions that we did at Intuit, a senior manager said it was interesting that they actually had to teach the teams how to persevere, that teams were willing topivot too early. It was interesting because, in the startup world, I encounter something different. In startups, people push through no matter what the evidence might state, or they’re scared to even seek the real evidence. In large enterprises, it’s exactly the opposite. They get a little bit of evidence, and then they’re ready to pivot to the next idea.

Intuit’s senior managers were like, “Actually, there are all sorts of ways that you guys can test ideas.” You need that perseverance. The real innovation is not going to come easy, so you have to teach the internal entrepreneurs that they really need to iterate as much as they can before they “give up” and go off to a different idea.

To get there, you have to come up with new ways to measure these teams so that, on a week-by-week basis, they’re making significant progress. It’s, “You will go talk to 15 customers in the next week,” rather than just making promises like, “Oh, yeah. By the end of the month, we hope to have a dozen downloads.” The mentoring is really about teaching entrepreneurial teams to push aggressively.

LR: Why do you think it is that in the established companies, everybody gives up and goes, “Okay, let’s just pivot”?

BC: That’s a good question. Here’s another example of a client that we’re working with, a large consumer website. The President and CEO were urging the teams to move faster, so you had these teams running experiments, then they’d go, “Oh, I don’t know if we can do this.” And it was this senior leader that was going, “No, no. You have to do this. Push faster. Move faster.”

So it could just be that, when growing up inside larger, well-established organizations, you’ve been mentally trained to sort of resist failure. Or if you fail too quickly, you’d better do something else where you might be able to succeed. Maybe there’s even a fear of persevering, because, like, how far do I want to push on an idea that might not succeed? Is that going to come back and ding my career? So a lot of people inside these large enterprises are still trying to figure out, “Is senior leadership really on board with this whole idea of failing fast?”

That said, I’ve always been impressed that, when bringing these ideas to large enterprises, they have entrepreneurial people there. They have people that like chaos, that like wearing many hats and are problem solvers. And actually inside these large enterprises are pockets of agile programmers and design thinkers. So they already have these skill-sets at the grassroots. Those are the people that we can help identify, and teach them a little bit more about Lean Startup and how they can start running experiments. And then how they can teach it and how they can become mentors.

Real cultural transformation is going to take a while, and it’s going to be difficult. It’s not a linear journey. It’s not from here to there. It’s not short. Because you’re going to have ups and downs, and you’re going to come across these obstacles, and you have to go out and find mentors, and you have to go and find allies—all in order to overcome the obstacles.

But I think the rewards at the end are pretty big, especially if you’re the one that’s driving it, right? You’re sort of an internal hero. Hopefully what it does is help your organization create real value for their customers and discover new business opportunities that result in real revenue. I think there’s a lot of upside. And the bottom line: My hope is that employees feel empowered to create new value for customers. I think it’s necessary for these big companies to survive, but it’s probably also necessary for these big companies to keep their best employees, as they feel empowered to not just follow the process inside their known market but to go out and discover new value.


What about the person with an entrepreneurial spirit who’s working in a large company? Can they make a difference on their own?

Recently I presented at a startup conference where I met a lot of people at the practitioner level who were asking that question: How do you get buy-in?  What I tell them is: Don’t wait for permission. Just start running your experiments. I’m sure you can find something that you can work on. Maybe it’s after hours, and maybe it’s beyond your existing scope. But once you start having evidence, you’re in a position to start training leadership because you’re not just asking for resources and money, but you’re showing them evidence that there’s something here that they should be interested in. Pretty likely the leadership is going to look at that and say, “Yeah, okay, I think you should continue that experiment.” And so that’s starting down the path.

My other bit of advice is to go find like-minded people. Run a meet-up inside the company, do a lunch-and-learn, do a regular happy hour where you get together with entrepreneurially minded people and start asking, “What are some experiments that we can run?” And you can start churning out evidence. Again, the leaders will start to take notice. And as a matter of fact, it’s likely that, when you reach out like that, you will find leaders that buy into this approach as well.

Eventually, I do think you have to have that C-Suite executive that says, “Yes, we are going to do things differently,” to get it to spread and to really get a cultural transformation. But you can get the whole process started just by going and doing it.


Lisa Regan is a writer for The Lean Startup Conference and creates content for individual companies—tech startups to design firms. She divides her time between the San Francisco Bay Area and Vienna, Austria, where she also writes on art and travel.




People building new software products tend to have a lot of similar questions. To get useful answers, we talked with Lauren Gilchrist, product manager at Pivotal Labs, a leading software development consultancy.

An edited, condensed version of our interview is below, and you can also catch Lauren in person at this year’s Lean Startup Conference, December 8 – 12 in San Francisco, where she’s leading a session on getting confident shipping products quickly so you can learn from users faster.

Ellen Sturm Niz: What do clients ask you most often?

Lauren Gilchrist: They often ask about growth problems before they’ve built anything. They have this huge vision in their head, and they just want to know how to get to that vision faster. So they ask a lot of questions like, “Should I build this feature first and then release that? Or that first and then release this?” Or, “Which of these seven different marketing strategies should I try?”

As humans, we tend to start with solutions. When we go to build a product, we don’t generally say, “What are the kind of problems that, for example, new moms in New York have?” Or: “I wonder which of the problems new moms in New York have is the most painful, and what’s the smallest thing that I can build to solve that problem?” Instead, people say, “What we need is a service that delivers diapers right to your door.” They start with that, which is the solution to a problem. They don’t start with the problem.

So a lot of questions that people come to me with look like this: They have a solution in their head, and they’re like “I’ve already got the truck lined up, and I’ve got the diaper provider, and what I really don’t know how to do is how to expand to 35 cities.” And we’re like, “But you haven’t launched anything yet.”

ESN: So what do you do?

LG: I tell them, “Before we deal with world domination, let’s back up.” I help people walk back up the ladder to get to: Who’s the user? What problem are you solving for the user? Does your proposed solution actually solve that problem—and how can you answer that? Then, how can you answer that faster?

ESN: So, if somebody came to you and they said, “Oh, I’m going to deliver diapers to moms, and I want to do it in 35 cities,” and you’re like, “Wow, you haven’t done it in one city yet.” What would you bring them in to do, and suggest that they do instead?

LG: Where I start first is this: “What’s the need? Help me understand why this is the problem that you’re devoting all of your time to solving.” That usually brings up a couple of different scenarios. One, people see a really big opportunity, whether it’s a marketplace or a need that’s not met. Or two, they’re dealing with a problem that they, themselves, are really familiar with.

There’s a woman named Allyson Downey who’s the founder of the company called WeeSpring. She has this great story about how when she was pregnant, she started asking her friends what kind of bottles they purchased, what kind of cribs they bought, what they needed, what they didn’t need, and it wound up turning into this crazy email chain of spreadsheets. She wound up launching a company a couple years later that was about getting recommendations for baby products from your friends. So, for her, that’s something where she really understood the need, and she wanted to build it. And she started very small.

Contrast that with someone who doesn’t have kids and says, “I’m going to solve diapering for every major city. I’m going to solve the problem that no one ever has enough diapers in their house.” They start with the classic business school case: here’s the market size, here’s how many people are going to have babies and diapers, and they work their way down to the three-year business plan. And nowhere along that journey have they even validated whether or not someone wants to receive a shipment of diapers within six hours. They haven’t talked to any moms, they haven’t potentially tried to make a quick MVP of like, “Hey, here’s a list of five moms, and we’ll just rent a car, go to the grocery store, pick them up, and drive over with the diapers.”

There’s this great story about Zappos when they first started, their MVP was actually that they took photos of shoes in retail stores and posted them. Then, if someone actually bought something, they would go buy the shoes in the retail store and ship them to the purchaser. That’s a great MVP. [This may be obvious, but it bears emphasis: It was a great MVP because it quickly and cheaply answered the question, “Will people buy shoes online?” Once they proved that people would, it became worth it to start building a more thorough system to do that. -Eds]

ESN: I love the story of Zappos. When I heard it, I thought, “That’s genius.”

LG: It’s so smart. It’s amazing. I worked for a startup called Yipit, and they did something similar. This was back when daily deals like Groupon and Living Social were really, really hot. They thought, “Oh hey, I bet it would be cool to see all these in one place, rather than subscribing to six different emails.” So, in three days, they built a website where you could sign up, and they kind of aggregated every single daily deal into their database. So, they would get up at like 3:00 in the morning, and put all of these things into their database, and then they’d send out the email. That was their MVP. And a year later, they actually still didn’t have a way to automate it. It was still all manual—there was a team that manually copied and pasted these daily deals into their database.

Obviously, they automated it finally. But for a year while they were still growing, and still getting their user base, they were building from the ground up and it was a cheap way of making it work without starting with, “Oh, what we really need is a web scraper that goes into a back end and can handle all these different cases.” Because they didn’t know if they actually needed to invest that money at first.

ESN: What are one or two questions you wish clients would ask you, but they never do?

LG: I wish they would ask, “How do I know I’m right?” As entrepreneurs, we start with solutions, and sometimes you don’t learn until much, much later that nobody wants it. It’s this double-edged sword, because to be an entrepreneur, you have to face down a lot of adversity, and you have to deal with people saying, “You’re wrong, and that’s crazy, and you’re never going to do it.” You have to power through that.

At the same time, left unchecked, the assumptions that you’re making that your solution is actually adding value to your users can grow and grow and grow, and it can make a failure much more expensive.

ESN: In other words: The belief you need in your product to overcome the adversities can cloud your judgement. So you don’t see when you need to pivot and do something differently?

LG: Right. It’s a tough balance. Those are the moments when it’s really helpful to come back and say, “How do I actually know that I’m adding value? How do I know that this product that I’ve built is solving a problem?” Without hard evidence, it’s really challenging. That ties back into, “How do I know that I’m right? How do I know that I’m adding value—and how I can actually validate that?” That’s something I wish people would ask more of.

What I teach people over the course of working together, usually, is that you can actually know whether something is adding value before it’s a fully baked, fully featured product. Which is really exciting. That learning and that validation that you get is so satisfying.

ESN: At The Lean Startup Conference, you’re going to talk about how releasing a product that doesn’t solve all of your customers needs at once is okay. Is that what you mean here? That you can get the information and the validation that you need with a less-than-perfect MVP?

LG: Absolutely. An easy analogy is when you release a product. The time period right before you release is kind of like going grocery shopping when you’re really hungry. You’re like, “Oh my god, maybe I really need the rack of lamb with goat cheese and lavender and honey. Or maybe I need some chips. Oh my god, I need a frozen pizza too.” And you’re sort of just crazily throwing things into your cart, because you are so ravenous that you don’t necessarily know what you need. You just need something.

Right before you release a product, you’re like, “Oh my god, what if we forgot the share on Tumblr button? We forgot the “Share on Tumblr” button. We totally need that.” Or like, “Oh my god, we can’t ship to London. What if we get a user in London?” You have this moment of absolute panic, because you’re about to push this thing live, and it may not be perfect. You may have overlooked something. It’s terrifying. It’s absolutely terrifying.

But what I’ve learned over time is that you can mitigate that panic. Just like you’d say in this same analogy, “Eat an apple before you go to the grocery store.” Same analogy you’re carrying out to the product side. You can release smaller versions of your product, or pieces of functionality. It doesn’t even have to be a functional product. You can put wire frames, or sketches, or something that’s totally fake in front of people to get feedback before you have to release this product into the world. And that helps you answer some of these questions about like, “Is this actually adding value, and how, right now?”

ESN: Right, is this “Share on Tumblr” button really going to make or break the product?

LG: Right. Not only is it going to make or break it, but do I even need it? So rather than saying, “Let’s put the button in just in case and delay the launch for a week,” you say, “We’ve already figured out that none of our users use Tumblr.” Or, “Well, we put a ‘Share on Facebook’ button in, and no one used that, so we’re just going to assume that we don’t need that.” Or, “We put a ‘Share on Tumblr’ button in a prototype and no one even asked what it did, or no one tried to click on it, so we can probably do without that.”

ESN: How do you know when you should delay a product because you overlooked something really important, versus when you’re just having that, “I’m hungry at the supermarket” moment?

LG: Ideally, that overlook moment isn’t coming at the point when you’re going to ship something. It’s coming before that, because you’ve been putting your product in front of people before your ship date. You’ve been interviewing users. You’ve been doing actual user testing sessions. That tends to come up earlier and some of the red flags are that no one’s using this feature, or maybe it’s a product that’s supposed to be inherently viral and no one is inviting their friends.

To go back to the diaper example, maybe everybody signs up but no one actually orders diapers. Or, nobody signs up. Or maybe you show this product to someone, and then you ask them how they would explain it to a friend—then they explain it like, “This is a new baby registry,” when you think it is a product where you could get recommendations from your friends about baby products.

When you’re like, “Huh. Whoa. People perceive this differently than I do,” that’s where you might decide, “Huh, maybe this isn’t ready to go, because people aren’t using it and talking about it in the same way that we think they should be, in the way that communicates that they understand the value in the same way that we do.”

ESN: I would think that entrepreneurs would fear that releasing something imperfect would affect the reaction of users. Do they think, I know people won’t like this because it’s not perfect so why release it yet? I’m just setting myself up to get negative feedback?

LG: Yeah, there’s a lot of fear. When you’re an entrepreneur, you have an enormous vision. You’re trying to disrupt an industry, or disrupt patterns. You’re trying to cause people to change their habits and change their behaviors. That’s a very big vision. If what gets released doesn’t match what’s in your head, it can be really hard to accept that users aren’t using the product for a fundamental reason, not because it doesn’t contain everything in your vision.

Another thing we get a lot of, particularly for projects that are for internal customers, is, “Oh my god, it didn’t solve everything.” You get this fear that everything is going to be judged on this one big release and reveal. That’s not actually true. There’s so much work that goes into a release and so much work that happens after it. But you get caught up in the big reveal moment, and that’s another thing that trips people up.

ESN: Does releasing an imperfect MVP ever kill a project? Does it ever happen where you say, “I knew I should have put that ‘Share on Tumblr’ button on there, and that’s why they hate it, and now everything is ruined!”

LG: But there’s so many ways to fix that. You can say—and this is so much about communication—you can always say, “Thanks so much for your feedback. We incorporated that. We listened, and guess what, now there’s a ‘Share on Tumblr’ button, and we put it out three days later.” The other thing that’s important to think about is that products are never done, they’re always evolving. They’re living, breathing, evolving beings, and it’s not as if you just push it out there, and then it sits there for two years, and you don’t change anything.

ESN: You can’t just sit back and say, “Hey, look what we did! I’m done and now we can just make money.”

LG: Right, exactly. You don’t put your feet on the desk and your hands behind your head and have a beer. You’re like, “It turns out we have all these users in the Philippines, how did that happen? I don’t know how we wound up there. Huh, that’s interesting. They can’t check out. We need to fix that.” And all of a sudden you’re like, “Are we a company that caters to a Filipino audience? Do we need two different sign-up flows?” You notice these trends, and all of a sudden you’re like, “Huh, that’s not what it was a couple of months ago. That’s not what is was three days ago. What is this evolving to?”

Going back to the launch scenarios, you learn from releasing something that’s imperfect, because you learn what problems you’ve already solved for people—and you learn what problems you still have yet to solve. You also learn more about what your users think. No matter what stage you release something, whether it’s literally sitting down and putting a napkin sketch in front of someone and saying, “What do you expect to happen when you click here?” Or whether it’s functional software that you release to millions of people, you’re still going to learn. The earlier that you can get something in front of people, the faster you can launch.

ESN: Can you give an example of a company that did release something that was imperfect in the early stages, that didn’t quite solve all the problems, but was good enough, and it worked out?

LG: I want to go back to that Yipit example, where they launched with like four daily deals rolled into one email, and they launched in one city. And then a couple of months later they launched in another city, because they were getting emails saying, “Hey, can you launch in Boston?” And you couple of weeks later, they launched in 10 more cities. And a couple weeks after that, they decided to add in categories, because people were like, “I don’t want all the daily deals, I only want spa deals.” It evolved slowly but surely. Now it’s a way to search for any discount for anything. But that’s not where it started.

ESN: What’s another question that you wish entrepreneurs or enterprises would ask you, but they never do?

LG: “Should I even build this?” I wish they would ask that, because validating the idea in the first place would save time and money and frustration on their part. They’re convinced that they should build it, there’s no question in their mind. But when you ask someone, “What’s the evidence that you should build this?”, they’re very taken aback. It’s like, “Wait, why are you questioning my vision?”

ESN: Because to them it’s so obvious. They know.

LG: Right. It’s so obvious to them. But a lot of times, there’s not a lot of evidence. I’ll give you a concrete example. I was working on a project where we were building a content management system, and we interviewed all the users, and we tried to understand what they needed to accomplish. It was a live video system, and often, the users needed to cut or reorder the segments for breaking news. They also needed to be able to see what everybody else is working on, and they needed to be able to edit things quickly.

You listen to that list of requirements, and you think, “Huh, well, clearly we need drag-and-drop to be able to reorder things quickly.” Building drag-and-drop is actually pretty time-intensive: It can be a week or two of development to build that kind of interface.

What we started with was this: Let’s put a box next to each segment, and that box will have a number in it, and that number will determine the order on the page. If they want to change the order, they can change the number on the box, and then refresh the page. We built that, we put it in front of them and we said, “Build a show on this tool.” Guess what? There wasn’t a single person who said anything about it being necessary to reorder segments faster.

So we validated that we’d solved their problem, A, which was needing to reorder segments quickly, and, B, we saved two weeks of not needing to build drag-and-drop, because we solved the problem in an easier way. In other words, we built the smallest feature that we could to solve that problem, and then we put it in front of them. That let us learn whether that solution solved their problem or not.

Had we learned that it wasn’t fast enough, we would have progressed to a drag-and-drop, or maybe another solution.

ESN: One of the other things you’re going to be talking about at the conference is about how too much empathy for customers slows product managers down. It seems counterintuitive in the startup world to be less empathetic to users. Can you talk a little bit about that?

LG: Absolutely. In my job, the process of finding out who the users are and what their problems are involves interviewing people and asking them a lot of questions and asking why a lot. The project I’m on, we just interviewed eight users at this company. We asked them about everything from their day-to-day habits, to how they use certain tools, what their frustration with those tools were, how they knew when they did a good job. You’re doing this psychological digging of, “What is it that’s really bothering you about this thing?”

You then have all that information in your brain as you start building solutions, so you’ve become incredibly empathetic with these people. All of a sudden, you’re like, “Oh my god, we can’t ship that, because it doesn’t have Bob’s drag-and-drop, and Bob really wants drag-and-drop!” It becomes second nature to want to solve all of their problems and want to help them and want to be this interesting savior figure, whose software solves everyone’s problems.

That’s where I really empathize with entrepreneurs who say, “I just want to fix it all.” But you can’t. You have to realize that people have some large problems and some small problems and some things that aren’t really problems at all. If you can fix one and make people really, really happy, then they’ll start asking for other things.

ESN: So, users will forgive you if you haven’t anticipated everything, but if you solved one of their problems really well, they will be happy with that. And they’ll give you feedback about other problems you could solve that you add later.

LG: That’s exactly it. There’s a Nobel Prize-winning psychologist, Daniel Kahneman, and he has this saying, “Nothing is as important as you think it is while you’re thinking it.” Which basically translates to “Yes, this is the thing that’s keeping you up at night, but your user hasn’t logged into your tool in two weeks and doesn’t remember it.”

ESN: You think Bob is secretly stewing over the fact that there’s no drag-and-drop, but really, Bob is on vacation.

LG: Bob is really worried about the fact that he has a flat tire, or his kid is sick, or he is hungry. There are so many factors in our lives, and the way that people use software is just one of them. And interestingly enough, if you don’t solicit feedback on your product in a way that is encouraging to get both positive and negative feedback, a lot of times you only hear the negative. Because what do people post online? Angry rants.

A lot of the positive reviews will get lost, and people will only talk about their frustrations. What you miss was that experience of watching them use it for the first time. Seeing them say, “This is great.” So, a lot of what we encourage people to do is be there at the point when 10 or 15 people are interacting with your software for the first time, and observe them, and ask them questions. You can watch their faces, you can watch their movements to understand whether or this is actually solving a problem for them.

ESN: So you should get really hands-on user feedback?

LG: Yes, and testing. There are two different phases: user interviews and user testing. User interviews are how you understand who your users are, what’s their demographic information, what are their behaviors, what are their needs and goals, and what is their motivation. And you sort of do that in a sprint, five or 10 people, and you’re like, “Okay, I’ve got all the information about these people and now I understand how my product might possibly add value.” Then you put your head down and you build for a little bit, and then you pick your head back up, and you go back to those 10 people for user testing.

You say, “Hey, here’s this thing. I’d like to watch you use it for the first time. Can you try to do this, and can you try to do that?” And like, “Oh, I saw you got stuck there, why did you get stuck?” In that moment, you get these amazing reactions. You either get, “This is the greatest thing ever.” Or you get, “I don’t get it.” Or, “What goes here? I’m confused.” Or like, “This says name, does it mean customer name or my name?” Like, Bob didn’t realize that he could drag-and-drop.

That’s ideally how you learn whether or not this adds value, and, also, what the other problems are. You’re doing it first hand, so when you launch this thing, you’re confident that you actually have solved somebody’s problem, and you also have a decent understanding of what the complaints are going to be. Maybe you’ll get some surprises, but you’re much more confident.

ESN: Is there any other advice you would give to product managers who want to get their services and products to users more quickly and effectively?

LG: First, before putting your own product in front of people, practice user testing on someone else’s product. Go to a friend and say, “I’m going to show you this product called Wikipedia. I want you to use Wikipedia in front of me to do these specific tasks.” User interviewing and user testing are really sort of awkward things, because you have to be able to sit back and watch someone call your baby ugly—and then just ask, “Okay, why is my baby ugly? Why do you think that?”

If you practice doing that on things that you are not emotionally invested in, it makes the habit and the practice a lot easier when you work on something that you do care about. Even if you do it among your friends or some coworkers and pretend to demo something that isn’t yours, you get that fear out of the way. It becomes more natural for you to ask those questions and to demonstrate something that is new to someone else.

Second, whatever fidelity you have, put it in front of someone. I have people that come to me and they’re like, “Well, I can’t show it to anyone because it’s a sketch, or it’s a wire frame, or it doesn’t actually work.” What I tell them is that you can learn something from whatever level of fidelity you put in front of someone.

For the content management project, for example, we started with just wire frames, and we put those wire frames in front of people, and we said, “Does this have everything that you need to do your job? Is this laid out in the same manner?” They said, “We need to be able to enter data.” So, we built a tool to let them enter data. We actually built the front end of the website and hooked it all up so that it almost was working. We let them type into it, and that’s was where we realized people were missing the save button, which is really bad.

ESN: That’s super frustrating to the user.

LG: It’s super frustrating, and we were able to catch all of that before we actually had the functional software. So, on the one hand, someone could possibly say, “It’s not ready yet. It’s not actually built.” But on the other hand, we were able to learn that faster, which means that by the time that we actually released the software to these users, we were able to validate in a day whether this was a tool sufficient for their needs.

So no matter what you have, and no matter how polished/unpolished, whatever, just put it in front of somebody, because you’re going to learn something, and that’s going to help give you feedback, and it’s going to help you answer some questions, which always helps refine your product.

Third, at the lab, we have a regular user testing schedule, so we actually bring in a couple of people every Thursday from Craigslist. Staff can sign up for slots and say, “Hey, I need to test this new flow that I’m working on,” or, “I need to figure out if someone is going to understand what this app even is, and can they explain it to a friend?” They have the opportunity to have someone just in the office. If you make it a habit where you don’t even think about, you don’t have to have this existential crisis of getting all these users, and, ”Who do we interview and when do we bring them in?” It’s just routine, and it’s there, and you can do it once a week. It becomes less scary. It’s all about reducing this fear of getting things in front of people faster.

ESN: Just put it in front of a couple of people and see if they do it.

LG: Right. For a product that I’m working on right now, we have a date range. Except rather than its having two date fields, one is a date and the other shows up with a button that says, “Add end date.” I was panicking, like, “I don’t know if this is the right way to do it. Are they going to understand that to add a second date in a range, you click the button and then you get the thing?” Everybody did it without a problem.

So my advice is always whatever is keeping you up at night, you can probably figure out by putting something in front of users.

By the way, on this project, the date range worked, but we have a button to add the customer, and the users didn’t know what that did. So now I have a new problem to work on.

ESN: Are there any other articles or information on user testing that you think would be helpful for people?

LG: There is a guy named Rob Fitzpatrick, who wrote a book called The Mom Test. He also has a bunch of videos. He’s the one that taught me how to phrase your user interviews and your user testing questions in a way that is not leading, that actually gets the information out of the people that you need to get it out of. That one was really useful to me.

Ellen Sturm Niz is a freelance content strategist and writer for The Lean Startup Conference.


By Lisa Regan    @mybldanube

The Lean Startup Conference, a three day gathering of entrepreneurs, is held at the Nob Hill Masonic Center and the Fairmont Hotel in San Francisco on December 8-11, 2013. (© 2013 Photo by Jakub Mosur)
The Lean Startup Conference, a three day gathering of entrepreneurs, is held at the Nob Hill Masonic Center and the Fairmont Hotel in San Francisco on December 8-11, 2013. (© 2013 Photo by Jakub Mosur)

Every time user experience expert Laura Klein gives a talk, she’s asked, “How do I get my coworkers to understand that what I do is important?”

Her insightful answer is below, excerpted from a recent interview with Lisa Regan for Eric Ries’s blog; you can read the full interview here. Laura is the author of UX for Lean Startups, creator of the design blog Users Know, and Head of Product at Hint Health, a software company working on healthcare affordability.

Lisa is a writer for both The How and The Lean Startup Conference.

Lisa Regan: So what are some of the questions people most often ask you, and what advice do you give?

Laura Klein: Here’s one question that I’m going to share with you not because it’s the most important one, but because it is the one that I get asked, I think, in every single talk I ever give that has anything to do with research. The question is, “How do I get people at my company to understand that what I do is important? How do I get them to see the value of User Experience research or design?”

The answer that I used to give, which I think was not very helpful but was how I felt, was, “Quit and join someplace that actually understands what you do, because they’re going to win in the end anyway.” But I think the right answer—assuming that first answer is not an option or that you just enjoy tilting at windmills—is to find a way to treat the other people at your company as themselves being users of your research. Just as you would with a customer, you need to understand their needs better.

First, you need to find a way to have them experience what their customers are actually going through. Often, a good way to do this is with video. So you would just go ahead and do that user research yourself, figuring out a fast way that you, on your own, can capture that information and then share those little snippets with the people that you need to convince. Nothing is more impactful than actually seeing what you are putting your customers through. Because it’s very easy to ignore that or not fully accept that there are problems with your product, but seeing person after person after person struggle with the same thing is incredibly impactful. So that’s an excellent way of doing it.

But I think that another thing to do is to understand why they don’t care about this. Is it because they don’t understand the value of it? In which case something like showing a video of people struggling with the product, and then showing video of people not struggling with it after it’s been fixed, can be incredibly helpful. Is it because they’ve had a bad experience in the past with research? I think then the answer is to help them understand ways to do research or design that aren’t what they’ve experienced in the past. Maybe they worked with a really big agency that charged them a huge amount of money and gave them really crappy results. I’ve run into this. Maybe they’re just like, “We don’t have time for this.” I hear this all the time, “We don’t have time for research.” At that point, you really have to explain to them that they don’t have time to not do research—because it takes a lot longer to fix it later than it does to fix it before it’s built.

LR: Do you think people just don’t want to hear what they already know to be true?

LK: There’s also that, and I think this happens a lot, honestly. This happens whenever people are rushing to get new features out, and they’re ignoring the old product. There’s, “We already know about the product problem. We already know about that. We’ve heard about that.” At that point, you really actually need to show them the impact that the bug is having on their bottom line. In that case, metrics and analytics are really important. “Hey, look, look at the funnel. You’re spending all this money to get people into the product, and they’re all dropping out here, and we’ve identified why they’re all dropping out here. This problem that you’ve been ignoring for a year, that I know you are totally used to, is costing you X amount of money or X number of users. This is a quantifiable problem.” If they’re real numbers-driven people, then that’s a good way of getting to them.

But again, I think the way to figure out how to change people’s minds within your organization is to understand them better. User experience people, we are trained to have empathy, or at least try to fake it, but we can understand. That is always our goal. We need to understand our users, and we need to understand our own problems. Apply that within your company, apply that to your engineers. I’m a big advocate for having empathy for your engineers. Understand what their needs are in terms of what you’re delivering to them, and they will be much better about implementing the things that you’re asking for. Understand and have empathy for your managers. Understand what they’re getting judged on and what their role in the organization is, and understand how to make them better at their jobs, and they will be much better about helping you do what you need to do.

This seems a little bit off the topic, but it’s around this concept of, how do I convince people that what I’m doing is important? I think the answer is that you give them not more deliverables or not more exact deliverables, but you understand what they need to see from you, and you deliver that. That’s a very user experience way of looking at things.

LR: What’s something that the product team would need to see? What’s an example of what that would look like?

LK: I just joined the team I’m working with as Head of Product a couple of months ago, and I’ve been working very closely with the engineering team. In fact, I have to tell them what to build, because part of the Head of Product’s job is to be sure that they’re building the right things. So the first thing that I did when I came in and sat down with them was I looked at how they’re currently working and I basically asked them, “What do you need from me to be successful?”

To get an answer to that, I would show them different levels of deliverables. For example, I would show them various works in progress. The last thing that I’m ever going to deliver to an engineering department is a pixel-perfect set of Photoshop mock-ups and just be like, “Here, implement this exactly.” That’s pretty much never the right thing. So I was like, “What level do you guys want to work at?” I would show them, “Here’s a task flow of when something happens. Then here’s some human-readable user stories that document everything that could possibly happen in this situation, and here’s a sketch with some notes on the side. Which of these–or is it some combination of these–will help you when you’re sitting down and writing the code? Also, by the way, I’m standing right here. I’m not away at a desk. I’m right here, a foot away. Literally turn around, tap me on the shoulder, and ask me a question.”

What we ended up with is that we use some combination of all of these kinds of things, including interactive prototypes sometimes, sometimes sketches with annotations, and sometimes test flows, and sometimes user stories, and sometimes sitting down and working through an error case together. We use a combination of all of these things to produce the different features. Depending on the feature, they get something different. Because it depends on if it’s a little thing, or a big thing, or a very complicated human interaction. They don’t just get the deliverable that I want to give them. It’s never that. They get the thing that they need in order to move forward and be successful on the first try.

The funny thing is that, with my team, I haven’t had to resort to showing them videos of the users. I probably won’t have to do that, because they are incredibly empathetic, and they see all the customer service emails that come in. They care deeply about the users already. So, I don’t have to convince them that users are important. But I think in a lot of engineering teams—not mine, luckily—but in a lot of engineering teams, especially at big companies, they can feel very removed from what the customer actually wants. So the design deliverables that you give them have to be quite explicit, because they are not necessarily going to have the framework for making good decisions about the user on their own. The right approach to that is to help them get closer to the user, because if they understand the user’s needs they will make better decisions on their own, because they will make them by thinking about the user. So I figure out what deliverables I can give them that will be the most useful will help them to make the right decisions.




Cindy Alvarez | Photo: Cindy Alvarez

In our “Now What?” series, startups ask real-life questions, and we find experienced entrepreneurs to offer deep, relevant advice. If you have a startup challenge, and you’d like insight from an experienced entrepreneur, let us know in this short form. – Eds

Minute Sitter’s issue

We’re in the process of testing the market for an iPhone app that helps parents connect with local, trusted babysitters and after-school care providers. Of the parents we’ve interviewed in our target market so far, this product resonates very strongly; most have rated the problem of not being able to find a babysitter a 7 or 8 out of 10. The problem is that it’s very difficult to find more interviewees. We’ve exhausted our friend network, and when we ask parents at the end of the interview if they know of others we could speak to, they always say yes, but it doesn’t progress any further. We’ve tried connecting with parenting Facebook groups to no avail, and it seems the next option is to pay for interviews, which we don’t want to do as this taints results. How do we find more interviewees?

About the expert, Cindy Alvarez

Cindy Alvarez is the author of Lean Customer Development: Build Products Your Customers Will Buy. She runs User Experience for Yammer (a Microsoft company) and has been helping companies build better products through intensely understanding their customers for over 14 years. Her background spans psychology, interaction design, product management, customer research, and lean startup tactics. She tweets and blogs.

About the reporter

April Joyner writes on business, entrepreneurship, and technology. She was previously a senior reporter at Inc., and she has also written for,, and OZY. She lives in Brooklyn, New York, and enjoys playing the violin in her spare time. Follow April on Twitter.

Interview with Cindy Alvarez, September 2014. Edited and condensed here.

April Joyner: What’s the first thing this startup should do?

Cindy Alvarez: The first thing would be to make sure that within their team, they’re aligned on what their hypothesis and target customers are. You need to boil things down to a sentence that is as simple as I believe this kind of person has this kind of problem, which could be solved in this way. And those have to be fairly specific—specific enough that you might think of a person that meets those criteria. A lot of times, I’ve found that teams aren’t actually in agreement on the hypothesis, even if they think they are. Once you have that, you basically have an implied audience in there. So then you start thinking about, “Who is that person, and where are they likely to be?”

Steve Blank has this great pyramid of needs. It’s basically a person who has a problem, recognizes they have a problem, has the ability to solve that problem, and has tried to put together a solution out of bit pieces. Those are the people you want to start with, because they’re the people who are the hungriest. This startup’s customer might be anyone who has a child, but that’s not necessarily your best market. That might be your eventual market, but the people you need to start with are the people who have met all those other criteria.

AJ: In this startup’s case, what might those criteria be?

CA: “Has a problem”: We define as people who have children who need care. “Who recognize they have a problem”: Something in their life is frustrating to them because they don’t have care. Maybe they’re not having date nights, maybe they’re not going to networking events, maybe they’re not seeing their family, or they’re not able to play a sport. You might have kids and not have a care problem. In that case, maybe your life would be happier if you had a babysitter and did this extra thing, but you’re not feeling it right now.

The next thing would be ability to make a change: In this case, probably someone who can pay. Your very lowest-income client is probably not the best place to start, because care is expensive in any situation. Then, beyond the ability to deal with it would be someone who has actually tried something—someone who has tried to find babysitters before, especially via some kind of online solution. If someone says, “Oh, yeah, I like to have a baby sitter,” but they have never made any attempt to acquire one, then they’re probably not the best customer for you.

Another thing that’s helpful is to do what I call a traits continuum, which is basically to write opposing traits, one on one side, one on the other, and figure out where you think people are. For example, for anything app-related, you might have from tech-savvy to not tech-savvy. Way over on the “not” side might be someone who’s never downloaded an app to their phone—probably not a good candidate here. A little further over might be someone whose significant other or kids put apps on their phone, but they don’t know how to do it. That’s still probably not the best market. Way at the other extreme is the kind of person who will try anything and download anything.

AJ: So once you know who your target customer is, what do you do next?

CA: A lot of times, if you have this list of traits, then you can say, “Oh, this person will be a great person to talk to.” You don’t necessarily need to identify every individual person you’ll speak to. But if you can read your list and it isn’t specific enough to make you think of even one unique person, you might be starting too wide. If you actually manage to tie it back to that one example person—like, “Oh, this sounds just like Pamela”—go talk to Pamela and say, “Do you use Craigslist? Do you use Sittercity? Do you use Care? How do you get babysitters today? What parenting groups do you subscribe to? Are there mailing lists? Are there children’s activity places that you go to?” Asking someone that level of information isn’t a strict customer development interview, but it’s saying, “Okay, we’ve identified a persona. Now let’s actually find that person and ask them where we should start looking.”

From there, try and figure out how can you convince that person to talk to you. In the software world, a lot of these pitches for interviews happen online. So, it might be constructing your email pitch. I typically recommend that people test it out first. Don’t blast out your email pitch as soon as you have it written. Send it to one or two people, maybe not even people who are your target customers, and say, “Read this, how does this sound?” A lot of times, the first draft email will not have the tone that you intend. A friend of yours might say, “This sounds arrogant,” or “This sounds too informal,” or “This sounds overly formal,” or, “This sounds like I’m not sure what you’re going to ask of me.” Other people are very good at picking up on those little weird bits of language that influence response rate. So I might write a pitch, send it to someone, get their feedback, change the language to take care of any issues, and then start sending it out to other people who I think are my real target customers.

AJ: To backtrack, since you mentioned email: when you’re brainstorming where customers might be, you’re actually gathering emails?

CA: It depends. There are some places where you might be able to contact people somewhat directly through the site, like LinkedIn or Quora, for example. That’s probably a less good option for a parenting-specific site. For that, the places where you find people might be, say, Parenting Mailing List. A mailing list is a place where you’d be able to get someone’s email address fairly easily, but something like a parenting forum is not so much. Generally, posting to a forum to say, “Hey, do people want to talk to me about my business idea?” is seen as sort of a negative thing. That’s along the same lines as advertising your product, and no one really wants that. In those cases, you may need to become a contributing member of that community first, invest that time until you get to know people. At that point, you have a little more social acceptability to ask questions. Or you can make friends with individuals and get their contact information that way.

With parents, the real world is a very, very good place to find them. Let’s say you have friends who have friends who are parents. Then what you are probably going to do is ask your friend to forward them an email. But your friend isn’t necessarily going to want to do that unless you’ve made it very clear that you’re going to be a good actor. I wouldn’t ask you to introduce me to a friend, and then say, “Go ask your friend to help me move my house.” What’s typically easy is to send an email to someone saying, “Hi, I’d really like to talk to parents in order to learn blah, blah, blah. You have a friend who I’d be particularly interested in talking to. Would you be willing to forward this email to them to make the request? I promise it will be no more than a 20-minute conversation, and if they don’t respond, I won’t continue bothering them.” There, you’ve established very clear parameters of what you’re asking for.


AJ: So, with this startup, it sounds like they’ve talked to some people, but they’ve exhausted their network. In that particular instance, how can they find other people?

CA: To be honest, what this smacks of to me is not that they’ve run out of people to talk to, but that there’s some other problem. Personally, as a parent, I think it would be pretty easy to get me to agree to an interview about this topic, because it’s a pretty painful thing when parents have a hard time getting care. The fact that people aren’t champing at the bit suggests to me that either the problem isn’t being pitched in a way that’s resonating, or that the request for conversation sounds onerous in some way. Maybe it sounds like it’s going to take a really long time, or they might be trying to contact people via phone who would rather use email, or vice versa. They might be trying to call people during the dinner hour. If you try to get anything out of me between 6:30 and 8:30 pm, I am not receptive.

What’s tricky is that people usually will not tell you what the problem is, so you have to do a certain amount of troubleshooting. If someone was like, “Hey, my friend really wants to talk to you about this,” and I kept saying, “Oh, yeah, I keep meaning to talk to her,” that’s a sign that for some reason, it’s not valuable enough to me. So the founders need to take a look and make some guesses about what that problem is. Is it that their pitch is not compelling enough? Are they communicating with people in a way that’s somehow off-putting, but they may not realize it? Are they communicating at people at the wrong time? Are they communicating with people in achannel that’s not common to them? Are they coming off somehow as advertisers? There are lots of things like this that might actually be an issue.

AJ: OK, so where should you account for these things? Is this something you need to think about right when you define who your target customers are?

CA: Whenever you’re going to interview customers, there’s a few things that you need to know about, what they value and what their limitations are. Typically, that’s what your network is for. You don’t necessarily have to be deeply embedded within your customer market, but you should at least be within arm’s reach of them. In this case, surely they have parents who are friends, or they have friends who have parents, who are friends. If they don’t actually have, say, five or six people that they can have a simple conversation with along the lines of, “What is the best way to reach you? Are there certain times that are a dead zone?” I think that’s a very difficult place from which to start a business, and I would recommend that they start making some friends who fall into that category.

You’re obviously never going to predict all of these things. But what you can do is constantlyiterate on the process. People who aren’t willing to talk to you may still be willing to answer a single question via email. If you say, “I’d love to talk to you about this solution,” and someone doesn’t respond, instead of continuing to try, you might say something like, “I’m just curious. We don’t have to have this conversation, but did I ask in a way that was inconvenient to you?” Or “Is there a way I could have phrased this better?” Or “Is there something I could have done to make this seem more appealing?”

That allows someone to give you a little bit of honesty without having to commit to a 20-minute phone conversation. You’re not going to get a whole lot out of one question that will stand in for a customer development interview. What you want to do is make sure that the next time you contact a parent, she actually says, “Yes, I want to talk to you.”


AJ: So this goes back to what you said earlier about developing a strategy to convince people to talk to you. Are there certain things that make people more or less likely to want to talk?

CA: Sure. You want people to talk to you; you want to recognize them as experts.

There are a few things that I can list that are turnoffs. One is when people feel like you’re trying to sell them something. You want to be really clear that you’re not doing that. At Yammer, I sometimes will start conversations with prospective customers by saying, “I’m not a salesperson. I couldn’t sell you this product even if I tried.” Eventually, of course, you’re going to ask people for their money, but when you’re doing the customer development interview, you want to remove that from the conversation. In fact, I don’t even like using the word customer with prospective customers, because I don’t want them in that buying mindset. So I’ll use words like you or your personal experience or in your life. I won’t say, “You seem like a prospective customer,” or “You might be a future customer,” because then I think they’re in that mindset of “At some point this person’s going to ask me for money.”

The second one is ego. People will say, “I’m a marketer with 20 years experience and blah, blah, blah.” If that’s the start of your pitch, then your email is basically saying, “Here’s a bunch of stuff about me.” So I’m not really convinced that you want my opinion because you’ve just spent a paragraph telling me all about you. I know people do this to gain credibility. It seems like a very logical strategy, and yet it falls flat on its face. I’ve gotten unsolicited customer development interviews where people go on and on about their credentials. I’m just like, “Ugh.” I barely read on.

I’d say the third one is an unclear ask for your commitment of time. Sometimes I will get a pitch and someone clearly wants to learn something from me about a product, but I don’t know what they want from me. Someone reading your email pitch or hearing your verbal pitch should have a very clear sense of what you’re asking for. I think people try to be polite—we think, “I won’t come right out and ask for things because that seems rude.” But giving multiple options is actually more of a burden because now I have different decision points to consider. The best pitches are very straightforward: “Can I talk to you for 15 minutes on the phone?” That is incredibly clear. I know exactly what I’m committing to, I know the medium, and I know it’s not going to take that long. I’m very likely to say yes to a pitch like that.

I think it’s generally best for you to pick a modality that you like, and offer another one as a fallback. Someone might say, “Look, I’d love to talk to you but it’s really hard to get me on the phone.” Then, you can say, “Can we converse via email or via chat instead?” At the last couple of companies I’ve worked at, I’ve had a large number of international customers. Between bad phone connections and accents on either side, either me not understanding them or vice versa, sometimes people will say, “Let’s just do Gchat.”


AJ: When you’re asking people to do customer interviews, has there ever been anything that’s surprised you about the process?

CA: Well, I don’t know if it’s surprising, but I think something that catches me off guard is mobile. At this point, more than 60% of emails are opened first on a mobile device. That is an incredibly short amount of space in which to make your point. If I’m looking at something on my iPhone screen, I’m seeing maybe two sentences. Somewhere in that two sentences, you have to hook me, and it has to be really clear how with one thumb I can hit reply and say yes. If not, it goes into the read-but-not-replied-to depths of my inbox. That’s purgatory. So I’ll write what I think is a really good succinct pitch, and I’ll send it to myself and open it on my phone. And I’ll be like, “Oh, the ask is way below the fold. This is terrible.” So I have to go back and cut more words.

The other thing is that people who are less tech-savvy have a very itchy spam filter. In talking to a lot of Yammer’s customers about exploring new features, a lot of the folks who are outside of technology are very suspicious that things might be some kind of spam or phishing. A lot of times, we’ll send one email to someone and see if it gets picked up on, and then send a few more, versus trying to blast people all at once. I’ve been very surprised sometimes by people who write back saying, “Are you a real person?” And I’ll read the email, and I’m like, “I don’t know what they’re responding to.” It seems completely legitimate—it’s from a real person, I’ve written it in a very human tone of voice, but something tripped someone’s “Maybe this is a phishing attack” filter.

I think one big thing is sending an email from an account that’s not a real name. People are suspicious of things that don’t come from humans. If you send from “MinuteSitter Support,” that’s not a human. If the email says it was sent from “Cindy, MinuteSitter,” that’s slightly better, but that might be someone selling me something. Another thing is, if you’re using a service like MailChimp or CampaignMonitor, sometimes the way the “sent from” line is rendered looks suspicious, like if it says, “From X on behalf of Y.” I’ve found that when things seem like they’ve been emailed through an additional domain, non-tech-savvy people don’t understand what that means—they just think it’s probably bad. At the startup level, I would just send emails from my personal account.

AJ: How can you tell whether you’re not asking people for interviews the right way, or if your product just doesn’t resonate with people in your target market?

CA: The easiest thing is to find some other person, even outside the target market, pitch them, and ask for feedback. So if the startup has an email drafted, forward that email to someone completely outside of their organization and say, “What do you think about this email? Would you be likely to say yes? What do you think about the people who wrote it?” It’s so valuable to have a friend outside the building for things like this. I just have a couple of friends, or people I’ve worked with in the past—at any given time I might send them an email and then follow up via chat, and be like, “Did you get that email? Was there anything weird about it?” Just the ability to do that saves so much time. If you don’t have that person, find that person in the startup community.

This is also something you could do with a quick survey. We’ve done this for feature work at Yammer, when we’re trying to ascertain whether the tone of our copy is positive for people. We might show a screenshot that has a bunch of copy on it, and then on the next page of the survey just ask a couple of questions like, “What did you think this was asking for? What did you think was happening in this step?” and see how people respond. We’ve definitely had cases where certain words had a certain connotation that people were picking up on. So we might show a screenshot, and then on the next page people would say, “Oh, I thought this was going on because this term seemed very negative to me,” and it’s often very surprising.

If they actually got people to respond to their pitch, and no one identified any issues with it, then I would move on to the next easiest thing to validate, which is, “Does this solution make any sense to their target audience?”

AJ: How do you validate that?

CA: Once people have agreed to talk to you, you want to know what they’re doing today. One trap startups tend to fall into is to ask aspirational questions. It’s typical to say something like, “Would you be interested in a service that does X?” That’s an almost useless question. The odds are that you’re going to get a “yes” answer, because frankly, it’s free to say yes. There’s no commitment involved. If you say, “Would you like a service that delivers chocolate to your house every night?” I’d say, “Sure.” Never mind that I’d have to pay for it, or that my health might suffer. The other thing is that most people have things that they wish they would do. If you’re asking about future behavior—”If you had this service, would you do X?”—people are just terrible predictors. It’s not just that they’re likely to say yes; they’re likely to be wrong.

Instead of asking, “Would you like to use a service like this?” you want to take it a step back and say, “Tell me about how you have found care for your children in the past.” Then you’re going to get answers like, “I’ve used this online service,” or “I’ve never used an online service,” or “I asked the person who lives next door because I know them, because I’ve lived next door to them for ten years,” or “I asked my friend who already has a babysitter how she found hers.” These are going to be useful bits of information, and you’re going to use them as a jumping-off point to figure out how, from that past behavior, you can shunt people into a new behavior.

A lot of times, by talking to people about what they’re currently doing, you can uncover their frustrations with what they’re currently doing. For example, someone might say, “Oh, I don’t have a problem getting a babysitter. I just ask my friend Joyce, who has a babysitter that she really trusts, and I just ask Joyce for that babysitter’s number.” The frustration might be that sometimes the babysitter’s already committed to Joyce, or frankly, Joyce is getting annoyed that you’re poaching her babysitter, and you don’t want to lose a friend over it, or that this babysitter’s great, but she doesn’t drive. Those little bits of frustration are where you can identify opportunities for providing a better solution.


AJ: How do you know you’re moving in the right direction, once you’ve tweaked your pitch and started talking to people?

CA: If you can get people to talk to you, you’re moving in the right direction. And once people have started talking to you, you should be listening for emotion. If you’re talking to people, and they’re very polite and mild-mannered the whole time, that’s a sign that you’re not really solving a big problem. I’ve never seen an interview case where people who were enthusiastic customers did not express some sort of frustration or excitement. Another big one is shame—people who feel like they ought to be doing something but they aren’t.

If you don’t hear the variation, if you’re not putting exclamation points in your notes anywhere, then you’ve got a bunch of polite people who probably won’t buy your product. I’d say if you talk to five people and none of them seem particularly enthused, then try talking to a different type of five people. It’s very unlikely that you’re going to strike out five times in a row, if you’ve really got a good market pitch.



AJ: Is there a certain number of interviews you need in order to figure out whether your product is resonating with people?

CA: The number of interviews people do is going to vary. A lot of times, I’ve said anywhere from 30 to 50 for this kind of scenario, where someone is just getting started. That person may be on the brink of making a big decision like, “I’m going to quit my day job,” or “We’re going to hire a full-time engineer,” or “We’re going to raise money.” Those are giant decisions. So 30 to 50 interviews are a lot, but if you’re deciding to quit your cushy day job and jump full feet into something, a lot of people want to have a sense of comfort. If you’ve already started a company and made those big decisions, then to some degree, you’ve already taken on that risk, so you might do fewer.

If you are able to very rapidly put out a minimum viable product and get people using it, then again, you might do fewer interviews because you’re going to be actually building the solution. Certainly, I’ve known people who’ve done five to ten interviews, but within the next week, they were able to put out a minimum viable product and get real customers using it. So they say, “Well, we are going to do a few interviews because now we’re actually watching people use the product, and people are giving us money,” which of course is the strongest possible signal.

If you have a startup challenge, and you’d like insight from an experienced entrepreneur, let us know in this short form. – Eds



Daina Burnes Linton | Photo: The Lean Startup Conference/Jakub Mosur and Erin Lubin

The Lean Startup method tells you to test your product idea with customers before you spend time and money building something you aren’t sure people want.

But how can you test a product without having the actual product? In her talk at the 2013 Lean Startup Conference, Daina Burnes Linton, CEO of Fashion Metric, explained how her company had done just that. As a result of their approach, Linton said, “Fashion Metric is today is very different from what our original idea was”—and she considered that a very big success. Here’s how Fashion Metric started:

We challenged ourselves to really understand the real problem the customer was experiencing…. We thought clothing shoppers—shoppers in stores and malls—might have a very difficult time deciding what to buy. Maybe it’s because they’re alone, and they’re not with their friends, and they can’t get their opinions. So, we thought, “Well, wouldn’t it be great if there was a mobile app, and you could take a picture of what you’re trying to figure out what to buy, and you can gain access to a personal stylist that can give you advice in real time and help make your purchase decisions?” Sounds like it could be a reasonable idea, right? Of course, friends and family always say, “Oh, yeah, that’s a great idea. Build that. That sounds awesome.” But we weren’t sure if we were solving a real problem, and so, we decided to really understand: Is this a real problem that customers are experiencing when they’re shopping in stores? So, we talked to who we thought our customer was all over the country. We went to malls in Los Angeles, New York City, and San Francisco. We asked, “What’s the biggest problem that you have when you’re shopping for clothes?” Very open-ended. What we found when we did this exercise was that not a single person—not one person—gave us the natural response that they had a hard time deciding what to buy. Not one person. So we almost built an app to solve that problem that didn’t exist.

By doing targeted customer research before building their product, the Fashion Metric team gave themselves 20/20 hindsight.

Of course, if they had stopped there, the company wouldn’t exist today. Instead, they looked carefully at the data they were gathering and realized that 90 percent of the men they spoke with said that finding clothes that fit was a problem. So the team dug deeper and learned that men particularly struggled to find dress shirts that fit well. With a clear trend around a real problem, Fashion Metric now had data to start building. Rather than code up a site, however, the team decided to validate their idea with an MVP that consisted of two parts: 1. a landing page asking for potential customers’ email addresses, and 2. follow-up phone interviews to gather customer sizing information. (Note that Daina is an engineer, so holding off on the coding was no small act of will.)

Before writing a single line of code, all we had was that landing page, and we learned a lot. We had an accelerated understanding of the problem; we were no longer building an app to solve a problem that didn’t exist; we understood the depth of the problem. We were able to see how far customers were willing to go to solve it. We were starting to see some trends in the data, to understand what questions we could ask, and whether or not it was technically feasible to solve the problem. You would think at that point, “Okay, great. Build something,” right? “Build the whole thing.” But we didn’t.

To find out what Fashion Metric did next, scroll down to watch or listen to Daina’s 12-minute talk. We’ve also included the full, unedited transcript below.

In the comments, we’d love to hear your best advice for finding relevant customers to interview before you have a product to show. Please tell us about the idea you were testing and how you found people. B2B and B2C ideas equally welcome! – Eds

Daina Linton is the co-founder and CEO of Fashion Metric, a company that builds technologies to increase sales and reduce returns in apparel e-commerce by improving fit accuracy. Fashion Metric’s comprehensive data-driven technology calculates a customers’ full compliment of measurements based on a seed of information provided by the customer. Fashion Metric then uses this algorithm output to fit customers in made-to-measure clothing or integrated on e-commerce platforms that carry “off-the-rack” sizes. A trained engineer, Daina pursued her degree while simultaneously holding several research internships at university affiliated hospital research labs. Eventually, she left a PhD engineering program at UCLA to parlay her experience in data analytics and image-processing methodologies and ultimately launch Fashion Metric. Follow her on Twitter.

Sarah Gaviser Leslie is a corporate storyteller and executive communications consultant in Silicon Valley. 

Jargon, demystified

Includes terms from the talk that we didn’t quote above. If there was a term you didn’t know that we haven’t defined, please let us know—we want to help! Also, if you have a better definition or an addition to a definition, shoot us a note.

MVP. An MVP, or minimum viable product, is an experiment that helps you quickly validate—or often invalidate—a theory you have about the potential for a new product or service. (Although often a stripped-down version of your product, an MVP is different from a prototype, which is intended to test a product itself and usually answers design or technical questions.)

The minimum concept is key because, in order to move rapidly and definitively, you want your experiments to include only features that help you test your current theory about what will happen when your customers interact with your product. Everything else is not only a waste of time and money, but can also cloud your results. The viable concept is key, too, in that MVPs have to actually produce test results you can learn from, which means it has to work on some meaningful level for customers.

The classic example is that if you’re trying to test the demand for a new service, you might do that by putting up a one-page website where people can pre-order the service before you’ve spent any time developing the actual thing or hiring people to make it happen. By the same token, if you’re testing a hypothesis around customer use of a new jet engine, then you have to make sure it will fly safely—in order for it to be truly viable, or feasible.

Landing page. A web page with a form to collect visitor info. HubSpot has an opinionated take on landing pages.

Ideation. Fancy word for brainstorming or idea formation.

Concierge. A type of MVP in which you manually fulfill a service that you’re thinking of automating in your product. A concierge MVP usually doesn’t require building much, if anything, and the high-touch interaction with customers lets you learn a lot.

Intelligence engine. Software that analyzes raw data to create useful insights.


Two New Experiments from The Lean Startup Conference Team

Post by Sarah Milstein, CEO of Lean Startup Productions

The 2013 Lean Startup Conference was a hit on several levels, with many people telling us it was the most valuable business conference they’d ever attended. To extend what we learned and help more people build and scale successful companies, we’re testing out two brand new events this spring:

Office Optional, a one-day conference on April 22 in San Francisco. One of our most popular talks at The Lean Startup Conference was my interview with Matt Mullenweg, in which he talked about Automattic’s unusual setup: They have no true central office, and everyone works from home. With partially and fully distributed teams on the rise—but few established standards for running them well—we see an opportunity to bring people together for a rich exchange. This conference hits close to home for us, as Lean Startup Productions has two desks at WeWork Soma, a coworking space, but no office of our own and team members scattered around a range of cities and time zones.

Quick Consulting, an evening event on March 27 in San Francisco. Inspired by the popularity of our experimental office-hours sessions at the 2013 Lean Startup Conference, we’re gathering more than 25 accomplished experts to bring you consultations for your business in a series of 15-minute, one-on-one conversations. This will be an intimate event designed to make it easy to meet other people and have high-quality, targeted discussions.

Below is more information on each event and some of the hypotheses we’re testing.

Office Optional. Although Yahoo famously recalled its remote workers last year, there’s no question that the workplace trend is toward more distributed teams. Driven by mandates for lower carbon emissions through reduced commuting, increased employee productivity and greater quality of life, paired with flexible technology, as much as 30% of the U.S. workforce now telecommutes. And yet, there’s still confusion over what to even call this kind of setup and who qualifies as a participant—let alone how to do it effectively. Our primary hypothesis is that with a growing number of remote workers in a variety of arrangements, innovative businesspeople—our core community—will find it valuable to come together to focus on solutions for successfully collaborating and managing from a distance.

(An in-person conference about remote work?! Is Henry Ford keynoting?! We get it. But we’ve found that certain discussions thrive in a live environment, and we believe this will be one of them. In fact, using face-to-face time really well is a topic for the conference—and we’ll help you make this event a highly useful place to meet with coworkers you don’t see often. For instance, we offer group packages that include meeting space for your team on the day before or after the conference. And, of course, we’ll provide a livestream of the talks; look for details on our site soon.)

How will we know we’ve succeeded? We measure event success in a number of ways including: whether attendees, speakers and sponsors say they would return again; whether our speakers get requests to talk at other conferences based on their presentations at ours; whether we’re able to cover our costs; and more qualitative feedback that we collect through interviews after the event.

We’ve just begun to announce speakers for Office Optional, with about a third of them now on our site and the rest to be revealed over the next couple of weeks. As you’ll see, we’ve got people from companies like Automattic and Yammer that are leading the way on distributed work, along with people from forward-looking service firms and non-profits that have terrific ideas we can all learn from. At the conference, which will have lots of Q&A with speakers, we’ll explore issues that include:

  • Building trust at a distance
  • Hiring, onboarding and training remote employees
  • Managing across time zones
  • Systems for communicating more and emailing less
  • Convincing colleagues to experiment with new tools
  • Tips and tricks for video calls, group chat, brainstorming software and more
  • Making in-person meetings very useful
  • Setting up satellite offices
  • The mechanics of successfully working from home

You can expect to leave with several big ideas for thinking about distributed teams, plus 20 or 30 concrete pieces of advice you can implement tomorrow. Early-bird tickets are available through this week, and they’re priced to let the biggest range of people attend.

Quick Consulting. At the 2013 Lean Startup Conference, we were amazed by how energizing and valuable mentors and attendees alike found the office-hours sessions. Our primary hypothesis is that a structured format with short, focused, one-on-one conversation will help eliminate awkward introductions and the distance between experts and attendees. Because it’s a bit tricky to ensure that all of the attendees have enough conversations to make the evening worthwhile, our secondary hypothesis is that we can create a format that meets everyone’s needs. That “everyone” includes Rackspace, which is hosting this event at their cool Soma space.

How will we know we’ve succeeded? The event is small—just over 25 mentors and 45 attendees—so in addition to a baseline survey in which we’ll ask whether people would join Quick Consulting again, we’ll be able to have follow-up conversations with a big percentage of participants after the event. Incidentally, we’re concierging the expert-attendee matchups, doing it by hand, which is no small task (the software we used for scheduling matchups at The Lean Startup Conference doesn’t have enough flexibility for the approach we’re taking here). If the overall format is a hit, we’ll look at automating this aspect of the event.

Our stellar list of experts have deep expertise in Lean Startup methods, entrepreneurship, user research, design, analytics, engineering management, PR, social media, startup law, corporate innovation, social-sector innovation, venture capital and more. Tickets for Quick Consulting cost $99, and there are just a few left. (If you can’t swing the standard $99 ticket price, we’ve set aside $30 scholarship tickets. Apply here to be considered.)

For both Quick Consulting and Office Hours, we’re continuing our work to ensure that we chose speakers by the most meritocratic processes possible and that we foster an atmosphere of lively learning. We hope to see you at one or both events, and we look forward to getting your feedback on whether our experiments are on target.

A New Approach to The Lean Startup Conference

Post by Lean Startup Conference co-hosts Sarah Milstein & Eric Ries

We’re excited to announce this year’s Lean Startup Conference, December 8 – 12 in San Francisco. Tickets are on sale now, at the best price we’re offering this year, so we recommend registering today. But that’s not really what we’re here to talk about. Instead, we want to focus on the themes of the conference.

This year, we’re defining the conference program as a series of hard questions entrepreneurs commonly face, and we’ll address them in our talks and workshops. Below is our initial list of questions. We’d welcome your input on which of these challenges feel most vital to you, along with additional questions you and your team are facing. You can leave ideas in the comments or email Sarah Milstein, co-host for the conference and CEO of Lean Startups Productions.

Very soon, after we’ve refined this list, we’ll post our call for proposals from potential speakers, seeking talks that can answer the questions we’ve collected. [UPDATE: The call for proposals is now open.] If you’d like to learn when we open the call, follow us on Twitter, subscribe to this blog, or sign up for the newsletter on our site.

Questions we’re aiming to address at this year’s conference, by category:

Experiments and Process

  • How can I ensure that meaningful customer feedback is included in our evaluation of new initiatives?
  • How can I get new products, services, and internal initiatives to market more quickly?
  • How can I design a good experiment–a minimum viable product–for services or internal customers?
  • How can I create a sandbox for innovation with my organization without putting my core business in jeopardy?
  • What can I do when a team proposes an experiment that might undermine our existing brand?
  • How can I experiment and iterate quickly on mission-critical products and systems?
  • How can I keep up team morale when experiments invalidate a lot of our ideas?
  • What can I do when I have a handful of customers who absolutely love our new product, but not enough to meet our revenue or impact goals?

Metrics and Accountability

  • How can I measure a new initiative before it has large numbers of customers or revenue?
  • How can I measure a value hypothesis and a growth hypothesis at the same time?
  • What metrics I can use to hold people accountable on projects that include extreme uncertainty?
  • How can I measure impact when financial metrics are not the bottom line (or not the sole bottom), such as in NGOs, non-profits and governments?

Teams and People

  • How can I convince my leaders and managers to support entrepreneurial methods?
  • How can I convince my co-workers and direct reports to use entrepreneurial methods?
  • How can I set up teams to ensure cross-functional collaboration?
  • How can I get internal services like IT, finance, legal, and HR to act like startups and serve entrepreneurial teams throughout my organization?


  • What does the culture of a high-performance, high-growth team look like?
  • How can I build a culture that serves existing customers and unlocks new sources of growth?
  • How can I best hire and train people who haven’t used Lean Startup methods before?

We look forward to your input on these questions in the comments or in email to Sarah—and we look forward to seeing you at the conference (register now for an honestly amazing deal.

A Few Surprising Facts about the 2014 Lean Startup Call for Speakers

Post written by Sarah Milstein & Eric Ries, co-hosts for The Lean Startup Conference

We’re seeking speakers we don’t already know for this year’s Lean Startup Conference, December 10 -11 in San Francisco. Our call for proposals is open now, and if you know already that you want to apply, jump to it (but read the directions first!).

If you’ve never applied to speak at The Lean Startup Conference before—or never even considered it before—here are a few things to keep in mind, some of them surprising:

You don’t have to be on the speaking circuit already. In fact, we’re psyched if you’re not a seasoned speaker. What we care about is the business experience you have to share, not the speaking experience. In other words, if you’ve applied Lean Startup techniques at your company, and you have advice or a story about your work that will help other entrepreneurs, we want to hear from you. While we’ll have a few speakers who are long-time Lean Startup experts, we’re primarily interested in case studies from people we don’t already know.

We don’t favor speakers we know personally. We aim to use the most meritocratic processes we can to find and evaluate speakers. That is to say: We don’t pick our speakers because they’re our friends or because they’re big names; we pick them because they have absolutely useful advice to share. In addition, we assess 95% of them via our call for proposals, which helps guide and standardize the submissions so that we’re comparing apples to apples (and not, for example, evaluating one proposal based on the recommendation of a mutual friend and another based on a video of the speaker from another conference).

Since we started focusing on fairness, a welcome though not surprising result is that we’ve regularly fielded a roster of excellent speakers that comprises more than 50% women and/or people of color. (In the past, the roster was nearly all white men, as those were the entrepreneurs Eric knew personally; they were very good speakers, too. But we now go much deeper into startup communities, finding speakers you won’t necessarily have heard elsewhere.)

Actually, we pre-select a very few. What about that other 5%? Full transparency: We do invite back particularly strong speakers from year to year, and we very occasionally invite entrepreneurs we meet out in the wild who strike us an unusually good fit for The Lean Startup Conference. We readily acknowledge that one of our best speakers in 2012 and 2013 was Sarah’s brother. We keep inviting him back because other entrepreneurs tell us they learn so much from him, not because he’s related to one of us.

We offer free speaker training. We do a lot of hands-on work with speakers to help ensure that your presentation really resonates with our audience. That includes, but is not limited to, both group and individual speaker training. If you’re a new speaker, this is a great opportunity to get some very good guidance (it’s also a great opportunity if you’re an experienced speaker; last year, some of our most avid trainees were our most accomplished presenters).

We’ll cover your travel costs. This year for the first time, we’re offering travel assistance for any speaker whose companies can’t cover it. We are ourselves a startup, and this represents a significant chunk of our budget, but we want to bring in speakers regardless of their own companies’ finances. (We also give all speakers a full Platinum Pass to the conference, but that ought to go without saying. If you register now to get the best price, and we subsequently pick you as a speaker, we’ll refund your ticket.)

Our call for proposals has a lot of directions—but the form itself is short. Don’t be daunted by the first page of the call, which has a lot of information that we ask you to take in. Those directions will help you submit a successful proposal, and they include links to some of our favorite talks from last year, for inspiration.

This year, we’ve come up with the questions our attendees most want answered—so you don’t have to do that part. All you need to do is figure out where you can offer relevant advice or a case study, and fill out the application form accordingly.

The call for proposals is open until 11:59p PT on June 12, and we encourage you to apply today. If you’re not interested in speaking but you know you want to attend the conference, register now for the best prices.