How to decide what product to build

Design is a process of making dreams come true.

THE UNIVERSAL TRAVELER

Learn faster. Dig deeper. See farther.

Join the O'Reilly online learning platform. Get a free trial today and find answers on the fly, or master something new and useful.

Learn more

LET’S PLAY A GAME. (I’m imagining the computer voice from the movie WarGames. GREETINGS PROFESSOR FALKEN...SHALL WE PLAY A GAME? Alas, I digress.)¹

How many people do you think are on the following product or feature teams?

• Apple’s iMovie and iPhoto
• Twitter
• Instagram
• Spotify

Hint: the number is definitely smaller than you think.

• Apple’s iMovie and iPhoto: 3 and 5, respectively²
• Twitter: 5–7³
• Instagram: 13 when acquired for $1 billion by Facebook⁴
• Spotify: 8⁵

We also know that the team that created the first iPhone prototypes was “shockingly small.”⁶ Even Jony Ive’s design studio at Apple—the group responsible for the industrial design of every product, as well as projects like iOS 7—is only 19 people.⁷ And we can surmise that this group is broken up into smaller teams to work on their own individual projects.

Figuring out what product you’re going to build is an exercise in working through the research you’ve gathered, empathizing with your audience, and deciding on what you can uniquely create that’ll solve the problems you’ve found. But it’s also an exercise in deciding how big the team is and who’s on it.

Jeff Bezos of Amazon famously coined a term for teams of this size: the “two-pizza team.”⁸ In other words, if the number of people on a team can’t be fed by two pizzas, then it’s too big. Initially conceived to create “a decentralized, even disorganized company where independent ideas would prevail over groupthink,” there’s some surprising science that explains why teams of this size are less prone to be overconfident, communicate poorly, and take longer to get stuff done. In actuality, that probably caps this team at or around six people.

Enter the work of the late Richard Hackman, a professor at Harvard University who studied organizational psychology. He discovered that “The larger a group, the more process problems members encounter in carrying out their collective work…worse, the vulnerability of a group to such difficulties increases sharply as size increases.”⁹

Hackman defined “process problems” as the links—or, communication avenues—among the members in a team. As the number of members grows, the number of links grows exponentially. Using the formula n*(n–1)/2—where n is group size—Hackman found that the links among a group get hefty very quickly (Figure 1-1).

Even though math wasn’t my favorite subject in school, let’s go through a few team size scenarios. Let’s start with Bezos’s recommended team size of six—assuming that two pizzas are appropriate for six people (although, I’ve been known to put away a whole pizza on my own from time to time):

• Bezos’s preferred team size of 6 people has only 15 links to manage.
• Increase that number to 10, and you already have 45 links to manage.
• If you expand to the size of where I work every day, Tinder—70 people—the number of links grows to 2,415.

But managing more communication links isn’t the only problem groups face when they increase in size.

Larger teams get overconfident. They believe they can get things done quicker, and have a tendency “to increasingly underestimate task completion time as team size grows.” In 2010, organizational behavior researchers from the University of Pennsylvania, the University of North Carolina at Chapel Hill, and UCLA conducted a number of field studies confirming these findings.¹⁰ In one of their experiments, they observed teams tasked with building LEGO kits. Teams with two people took 36 minutes to complete the kit, while four-person teams took over 44 percent longer.

But the four-person teams believed that they could complete the LEGO set faster than the two-person team.

That’s why the notion of the two-pizza team is so powerful. It’s a simple concept that’s easily understood by anybody within your organization, and can be used to combat the “let’s throw more bodies at the problem” mentality that some organizations might be used to using.

OK, so we’ve figured out how big your team should be. But who should be invited to the party?

Everybody loves to be in product meetings. Especially when you’re in the deciding phase of deciding what to build.

Even Steve Jobs loved being in the room during this phase. “He told me once,” said Glenn Reid, former director of engineering for consumer applications at Apple, “that part of the reason he wanted to be CEO was so that nobody could tell him that he wasn’t allowed to participate in the nitty-gritty of product design.”¹¹

Treat this process like you’re the bouncer at Berghain nightclub in Berlin.¹² (Hint: it’s practically impossible to get in if you don’t speak German. And even then, Sven the bouncer, “a post-apocalyptic bearded version of Wagner,” enforces an obscure dress code that nobody can seem to crack.)

So, who’s in the room together? How much do they know about the pains you’ve found? And how do you frame the discussion?

At this point, you should have everyone who’s going to be involved in the creation of the product on the team. An example of this could include:

• The product designer or product manager (depending on how your organization is set up, and if you’ll be working with someone else who will be designing the product).
• The engineer(s) with whom you’ll be working to build the product—typically frontend and backend.
• A representative from the team that will be launching and promoting the product; this could be someone from marketing or public relations to create a feedback loop between what will be promised to your customers and what your product is actually capable of doing.

While at KISSMetrics, Hiten Shah structured these teams with

…a product manager, a designer, and an engineer. Sometimes it’s multiple designers, multiple engineers, and sometimes it’s an engineering manager.

At times it can even be, sometimes, someone from marketing, if that makes sense, or even someone from sales. I mean, we have tried different methods. I’d say for different things, small things, big product releases, a whole product, it’s going to be different and for the stage of the company it’s going to be different.

Party Like It’s 1991

Regis McKenna had something to say about this process. When he saw how fast technology was changing society in 1991, he realized—like our friend Neil McElroy at Proctor & Gamble—that a new role would need to be formalized. This person would be “an integrator, both internally—synthesizing technological capability with market needs—and externally—bringing the customer into the company as a participant in the development and adaptation of goods and services.”¹³

If your eyes glazed over reading that, well, you should read it again. Because McKenna was responsible for launching some of the hallmarks of the computer age: the first microprocessor at Intel, Apple’s first PC, and The Byte Shop, the world’s first retail computer store. Oh, and one more thing: he was the guy behind the “startup in a garage” legend first made famous with Apple’s early days.

So, did you read it again? Did anything seem familiar?

Hey, he’s describing you!

You’re the product designer. The integrator. You’re the customer’s champion, their expert, their advocate.

This process requires you to lead your team through the research; to propose product ideas to eliminate your customer’s pain or find their joy effectively.

That, of course, means that everybody involved in building the product must be intimately familiar with the research that’s been conducted on your audience.

Take the opportunity as an “integrator” to build on your strengths as a team: what innovative technologies and design can you apply to the problem at hand? Even better, what can you and your team uniquely build for this audience?

I thought Josh Elman (Greylock Partners, Zazzle, LinkedIn, Facebook, Twitter) had a great insight on this part of the product creation process:

The first thing is you have to trust your team. I think that sounds obvious, but it’s much harder in practice. I think a lot of structures and processes are built on the fact that there isn’t innate trust…next, get your team’s help in how to solve the problem. The team knows what they can build. The team knows how it can be developed. The designers know what kinds of things are designable and natural in the product and what kinds of things are not. All of this matters.

Don’t forget the Pain Matrix (Figure 1-2). What are the observations you made that fit into the upper-right quadrant where there is the most acute, frequent pain? How can you build your customers’ dream product? What are the pains that you’re uniquely capable of solving?

 

The Pain Matrix is the perfect piece of collateral for when you’re hashing out what to build. This document becomes a communication device, an advocate for your customers. Everybody can see it and you can back it up with your data. Bonus points for direct quotes from your research.

“The thing to focus on is that yes, 100 percent of your users are humans,” Diogenes Brito, a product designer most recently at startup Slack, reminds us. “While technology is changing really, really rapidly, human motivations basically haven’t at all. Like Maslow’s hierarchy of needs, that’s still the same. Designing around that, the closer you are to the base level of what humans desire, the more timeless it’ll be.”

To reiterate: don’t lose sight of the actual, observed, tangible pains and joys that you’ve researched. Resist the temptation to delve into hopes and dreams. Just throwing an “MVP” out into the wild to “validate” something you spend time building is a waste of time, money, and talent.

You’re better than that.

Now, all you have to do is keep everybody focused.

Keeping Everybody Focused

There’s always a big problem when the club-like euphoria from a product meeting starts to turn focus into chaos. How do you keep everybody on task and debating healthily?

I highly recommend a whiteboard for idea collection and harvesting. This serves three practical purposes:

• It’s difficult to remember what was said. You don’t want good ideas getting lost simply because there were too many thrown around the room.
• It allows you to be visual. Not all ideas can be verbally explained; a low-fidelity medium allows anybody to sketch the central core of the idea without unnecessary detail. This allows your team to get ideas out of their head on an equal playing field.
• It lets you take advantage of the natural tendency for the group to forget which idea was contributed by whom. This naturally allows the best ideas to float to the top and the worst ones to sink to the bottom. It’s hugely beneficial, especially if the group has a lot of ideas. The key here is to avoid attaching names to ideas, so you can avoid hurt egos and the so-called not invented here syndrome. Called the Cauldron, this was a technique used by Apple—sometimes even with Steve Jobs in the room. According to Glenn Reid, the former director of engineering at Apple, the Cauldron “let us make a great soup, a great potion, without worrying about who had what idea. This was critically important, in retrospect, to decouple the CEO from the ideas. If an idea was good, we’d all eventually agree on it, and if it was bad, it just kind of sank to the bottom of the pot. We didn’t really remember whose ideas were which—it just didn’t matter.”¹⁴

There’s also the benefit of timed techniques, like one used at online publishing startup Medium. With the right group of people in the room, the problem that needs to be solved is defined and “you have two minutes to write down as many ideas as possible [to solve it],” director of product design and operations Jason Stirman told me. “Then you have five minutes to put the ideas on a whiteboard and explain them. Then you have another two minutes to add to ideas…the end result is you just get as many ideas as possible. So we do that a lot here. We brainstorm a lot.”

The “Working Backwards” Approach

There’s another technique used by Amazon that’s particularly powerful. Known as the “Working Backwards” approach, this technique calls upon the product owner to literally write a future press release for the product—as well as fake customer quotes, frequently asked questions, and a story that describes the customer’s experience using the product.

In your case, this could be a future blog post that you’d put out about your product or feature instead of a press release.

What’s particularly unique about this technique is that this document involves every part of your organization that’s required to make the product successful—not just product and engineering, but marketing, sales, support, and every other part of your company. In other words, it forces you to think about all of the aspects that can inform your product.

Werner Vogels, Amazon’s CTO, describes the rationale behind the process:

The product definition process works backwards in the following way: we start by writing the documents we’ll need at launch (the press release and the FAQ) and then work towards documents that are closer to the implementation.

The Working Backwards product definition process is all about fleshing out the concept and achieving clarity of thought about what we will ultimately go off and build.¹⁵

According to Vogels, there are four documents included in Working Backwards:

The press release

What the product does, and why it exists

The “frequently asked questions” document

Questions someone might have after reading the press release

A definition of the customer experience

A story of what the customer sees and feels when they use the product, as well as relevant mockups to aid the narrative

The user manual

What the customer would reference if they needed to learn how to use the product

This all might seem like a lot of frivolous upfront work, but the method’s been used at Amazon for over a decade. And if you use it in conjunction with the Sales Safari method outlined in Chapter 2, you’d be hard-pressed to find a more customer-centric approach to building products. That way, you’ll be working on ideas that have their foundation in what real people need, as opposed to coming up with ideas that you try to plug into an amorphous audience.

At the center of Working Backwards lies the press release. A document that should be no longer than a page and a half, it’s the guiding light and the touchstone of the product and something that can be referred to over the course of development.

“My rule of thumb is that if the press release is hard to write, then the product is probably going to suck,” writes Ian McAllister, a director at Amazon. “Keep working at it until the outline for each paragraph flows.”¹⁶

Amazon’s view is that a press release can be iterated upon at a much lower cost than the actual product. That’s because the document shines a harsh light on your answer to your customer’s pain. Solutions that aren’t compelling or are too lukewarm are easily identified. Nuke them and start over. All you’re working with at the moment is words.

“If the benefits listed don’t sound very interesting or exciting to customers, then perhaps they’re not (and shouldn’t be built),” McAllister writes. “Instead, the product manager should keep iterating on the press release until they’ve come up with benefits that actually sound like benefits.”

So what does this press release look like? Thanks to McAllister, we have a very specific outline of the documents Amazon uses in their product meetings:

1. Heading: this is where you announce the name of the product. Will your target audience understand its meaning? Will they be compelled to learn more?
2. Subheading: declare in one sentence who your product’s target market is, and how they’ll benefit from it.
3. Summary: summarize the product and and its benefits. McAllister cautions that there’s a large chance that your reader will only make it this far, so “make this paragraph good.”
4. Problem: this should be easy since it’s been the focus of your customer research. What’s the problem your product solves? What are the pains your customers are experiencing that justify this product’s existence?
5. Solution: how does your product annihilate your customer’s pain in the most frictionless way possible?
6. Quote from a Spokesperson in the Company
7. How to Get Started: how does a customer take their first step into the larger world that’s your product? Describe your ideal first step that provides an immediate benefit.
8. Customer Quote: what would your ideal customer say after they’d had their pains destroyed by your product?
9. Closing, with a Call to Action

I think that this process—however laborious—endures at Amazon because it provides such clarity about what the product is going to be, and how it’s going to help your customers.

“Once we have gone through the process of creating the press release, FAQ, mockups, and user manuals, it is amazing how much clearer it is what you are planning to build,” writes Vogels. “We’ll have a suite of documents that we can use to explain the new product to other teams within Amazon. We know at that point that the whole team has a shared vision on what product we are going to build.”

Create a Product Guide

Before you finish defining what product it is you’ll be building, you’re going to want to leave with some paper in hand.

I love Cap Watkins’s approach. The vice president of design at Buzzfeed and former Etsy senior design manager keeps his team on task after the meeting by creating a key internal document. We’ll call this a product guide for the sake of discussion:¹⁷

At the end [of the product definition meeting], leave with:

What you’re doing.

Why you’re doing it (problems you’re trying to solve).

What success looks like (quantitatively and qualitatively).

The product guide helps you “keep yourself and your team focused and prevent design creep: if it doesn’t solve the problem or meet the goals, it doesn’t go into this version.”

The only two elements missing from this approach are who’s responsible for what pieces of the product, and when they’ll be done.

Pre-dating Steve Jobs, Apple created a rule for every project they undertake: the “Directly Responsible Individual,” or DRI. It’s a simple yet effective rule. By placing one’s name next to a task-to-be-completed in front of the entire company, you can be sure that individual feels more responsibility to perform.¹⁸

In your own internal product guide, use this rule and make sure that every line item has a DRI. Assign someone (or recruit volunteers) to complete each task. Make sure this is in place before you break up the meeting. Include it in your guide’s circulation.

It’s not enough to put people’s names next to line items. You and your team need dates, too. Do you need to do research for technological or data considerations? Do you need to build a prototype to see if a particular interface component is possible? Did you misinterpret something in your audience analysis? What can be executed upon immediately to challenge any outlying assumptions? Who needs to see what and when (other product teams, clients, bosses)? At what fidelity?

By giving every task a deadline, you’ll maintain momentum even when the initial enthusiasm of creating something new dies down.

But how should you build in accountability? How do you race against the clock to get something valuable out to your customers?

Customer retention company Intercom—whose clients include Shopify, InVision, and Rackspace—gives their product teams weekly goals to hit. “We believe you can achieve greatness in 1,000 small steps. Therefore we always optimise for shipping the fastest, smallest, simplest thing that will get us closer to our objective and help us learn what works. All our projects are scoped into small independent releases that add value to customers.”¹⁹

At AngelList, Graham Jenkin places a similar pressure on himself and his team to execute on a new product as quickly as possible—with a bias toward small chunks. “We’re always thinking about how are we going to execute this by the end of the day, or ‘how are we going to get this done this week? Can we get it done this week?’ If we can’t, maybe it’s the wrong problem to try to solve.” His approach is more of a rolling set of needs, rather than standing, arbitrary deadlines.

And my current employer, Tinder, holds product update meetings every Monday, Wednesday, and Friday—but only if there’s something to discuss. These meetings typically consist of the product team gathering feedback from each other on their own projects and asking for critiques or help solving any design challenges. Every Monday is a roadmap meeting where engineering, product, customer support, and marketing leads get into a room and update each other on the progress of their projects, while alerting the group if something new needs to be addressed.

Whatever the setup is at your company, remember that you’re the integrator. So be the leader. Be the customer’s advocate. Don’t settle for hand-waving and bravado.

You’re better than that. More talented. And probably better looking.

Shareable Notes

• Defining what to build starts first with who’s in the room.
• When choosing who’s in the room, follow Jeff Bezos’s “two-pizza team” rule: your team should be small enough that two pizzas could feed them. Typically, this comes out to about six people.
• Only allow team members through the door if they’re educated on the research you’ve conducted on your audience. Use the Pain Matrix liberally.
• Whiteboards are your friend. They help you remember what was said, allow you to be visual (sketching together brings teams together), and disassociate ideas from their inventors. This allows the best ideas to win without any regard for who invented them.
• Amazon’s “Working Backwards” approach can help you pinpoint the product ideas that’ll solve your customer’s pains, versus creating something that’s too flashy or uninspired.
• Leave this meeting with a key document: the product guide, which outlines what you’re building and who’s responsible for doing it.

Do This Now

• Re-examine the knowledge your team has of your audience. Encourage them to study up on the research you’ve done so they can make more informed product decisions.
• Think about how your product can really make your customers happy. Are you really bringing them joy? Are you truly able to alleviate their pains and satiate their needs?
• Take stock of how your team conducts meetings and makes decisions. See if any of the techniques mentioned in this chapter can help you make more realistic decisions in a smaller time period.