Designing with Data

Chapter 1. Introducing a Data Mindset

Data as a Trend

DATA. THIS SHORT WORD has captured the imagination of the media. Every day, another news story breaks that preaches the power of “big data,” discussing the value of data for business, data for adaptive technology experiences, or data and marketing. It’s clear that regardless of the application, data is a very hot topic and the currency of the day.

It might feel like using data is big news now, but the truth is that we’ve been using data for a long time in the internet business. Data in the form of digital content and activity traces is at the core of internet experiences. For the past 20 years, we’ve been inventing new digital experiences and re-creating physical world experiences in the digital world. Sharing photos, having conversations, finding love: activities that we perform in our daily lives have all become digital. Being digital means we can log and track these activities with ease. Digital interfaces have made data collection so easy that now our biggest challenge is not access to data, it’s avoiding the false belief that data is always good, and recognizing that interpreting the data and deriving meaning from it is itself a challenging task. In other words, the ease of gathering data can lead us to be lazy in our thinking, resulting in erroneous conclusions if the data quality is low or unrepresentative or the data analysis is flawed.

There’s more potential here than collecting any and all data, of course. The “digital revolution” and the internet as a platform mean we can also run experiments to collect data that allows us to compare one experience to another. We have the potential to run many experiments, sometimes concurrently with many users at once—a practice that has been called “experimentation at scale.”

And that leads us to why the three of us wanted to write this book. We had two key reasons. First, so that more people with a user-centric orientation enter into the conversation about data collection, data quality, and data interpretation. And second, so that those who wish to can apply the information we share here and more effectively leverage data in their design work. We hope that you are able to use the information in this book to your benefit and that this will in turn further the practice of bringing data and design closer together.

Beneath the complex and ever-evolving world of experimental design and statistical analysis, there are some basic principles that are surprisingly powerful and very important to understand. Our aim is to give you a framework for thinking critically and carefully about the design of experiments, and to help you avoid the trap of just being excited about data for data’s sake. We want you to be excited about collecting and analyzing the right data, in the right way, with the right framework so you can maximize your understanding of what is important in your context.

When working with data and experimentation, you can’t take a “one size fits all” approach to your work. Depending on the nature of the problem you are trying to solve or the stage of problem solving that you happen to be in, you might employ different methods or techniques and take into account different kinds of constraints in the scope of your solution. We think it’s important to always be aware of this greater landscape in which you might work and understand how the particular method or approach you are using might have different pros and cons. We like to talk about this as being aware of “where you are” and “what is possible” in the landscape of different design activities you could be engaging in (Figure 1-1).

In this book, we focus on A/B testing because we believe that as a methodology, it has seen the least collaboration between design and data, but that this collaboration between design and data has the most potential for impact in A/B testing. However, A/B testing is just one tool in your data toolkit. It can extend the value of other methods you might already be engaging with, and it’s a foundational and versatile method that can be applied successfully to many different design stages and for many different kinds of design problems.

Figure 1-1. Charting the landscape of design activities.

As we stated in the Preface, we expect data scientists, designers, user researchers, and others to be involved in the conversation around trying to understand user behavior. We believe that applying data in the design process is definitively not about replacing the things that design processes and designers do well already. It’s about helping designers extend the value of those things by offering another way to look at the impact of the design work on users. A/B testing can’t answer all questions, but it can answer certain kinds of questions that other methodologies and practices cannot. In fact, we think you’ll find that working with A/B testing is actually quite similar to other evaluative design processes. When applied correctly, it is creative, iterative, and empowering. This book will help you get started applying A/B testing in this way.

Three Ways to Think About Data

Before proceeding, we’d like to spell out some differences we have perceived in how data and design have been positioned in the industry. The three terms we want to introduce are data driven, data informed, and data aware.

The terms data driven and data informed might already be familiar, but we have coined one additional term—data aware. One of the best descriptions that we’ve ever seen on the difference between data driven and data informed comes by way of Andrew Chen, a popular blogger who writes about online marketing. In a well-referenced post entitled “Know the Difference Between Data-Informed and Data-Driven,”¹ he explains that “the difference [...] in my mind, is that you weigh the data as one piece of a messy problem you’re solving with thousands of constantly changing variables. While data is concrete, it is often systematically biased. It’s also not the right tool, because not everything is an optimization problem. And delegating your decision making to only what you can measure right now often de-prioritizes more important macro aspects of the problem.”

Taking this perspective on, here are our guiding definitions of these terms.

Data-driven design implies that the data that is collected determines (in other words, drives) design decisions. In some instances, this is the right way forward. At times, the types of questions a team is asking can be definitively answered by collecting data from experiments. The outcome of their data collection maps directly to a clear best design decision.

You can be data driven if you’ve done the work of knowing exactly what your problem is, what your goal is, and you have a very precise and unambiguous question that you want to understand. This also assumes that your methodology and measurements are sound, and that the type of question you want to answer is one that data can determine. It relies on a keen understanding of the types of pitfalls that data can bring, and taking steps to remediate those pitfalls (Figure 1-2).

Figure 1-2. Laying out the relationship between “data driven,” “data informed,” and “data aware”—data-driven answers well-targeted questions. where data alone can help drive decision making.

In some instances, however, your design decisions may be more nuanced and the data might suggest an answer that is not cut-and-dried. This is what we call data-informed design, where a team takes data as only one input into their decision-making process. In this type of design, the output may not be a clear choice but may perhaps result in setting up another iteration or investigation. This is when more research may need to be done, different kinds of data gathered, and/or an informed creative leap taken.

So adopting a data-informed perspective means that you may not be as targeted and directed in what you are trying to understand. Instead, what you’re trying to do is inform the way you think about the problem and the problem space. You might answer some questions along the way, but you are informed by data because you’re still iterating on what the problem space is within the goals that you have. This is a slightly more creative, expansive, and critically iterative space. You can’t be data driven without thinking about your problem space in a data-informed way at some point (Figure 1-3).

Figure 1-3. Laying out the relationship between “data driven,” “data informed,” and “data aware”—being data informed allows you to understand how your data-driven decisions fit into a larger design space of what can be addressed.

Finally, we introduce and use the term data-aware design to underscore the fact that the design process is a creative one, where design decisions need to be taken not just from data but back to data collection practices—that how a system is instrumented, that what data types are being captured and how they are combined, is itself a design problem. In our view, designers and data scientists need to work with developers and business strategists to actively design systems so that the right data types are collected to address the right questions. We believe that designers have an important and unique viewpoint on the design of experimental hypotheses for collecting data to test design assumptions.

In a data-aware mindset, you are aware of the fact that there are many types of data to answer many questions. If you are aware that there are many kinds of problem solving to answer your bigger goals, then you are also aware of all the different kinds of data that may be available to you. You’re constantly questioning how you might best approach your goal. This is a more strategic way of thinking about how data can inform what you need. Again, you can’t get to the data-informed stage if you haven’t properly worked through the considerations of being data aware (Figure 1-4).

Figure 1-4. Laying out the relationship between “data driven,” “data informed,” and “data aware”—being data aware means recognizing that there are many related questions and also many related kinds of data that you can draw on to answer and influence a variety of questions.

We think of embarking on a data-driven process like being on a platform about to board a train (Figure 1-5). The train is on the right tracks already (it’s reliable, and where it’s going is fixed, determined, and repeatable). You have confidence that you’re on the right train and that you’ll be going in the right direction. There isn’t much more problem solving involved. You’ve already done all the due diligence and work (defining your problem, goals, etc.) to know exactly where you want to go and now your data can answer specific tactical questions. Data is a source of truth. (This is an application of data.)

Figure 1-5. A train on tracks is a good metaphor for data-driven processes: you know the approach you are taking, your approach is reliable, and what you are going to find out is understood, directed, predetermined, and repeatable.

When you’re data informed, you’re in the railroad station but there are many trains on many tracks that could go many places (Figure 1-6). You’re trying to decide which train to get on by thinking about where you want to go. Being data informed is about informing how you think about the problem. This is a creative and highly iterative process by which you learn about your problem space. (This is a discipline and a practice around data.)

Figure 1-6. A train station is a good metaphor for being data informed: you know there are various trains, and they are likely going to different places. You are aware there are options and there are mechanisms for finding out which is the right train for you. There is less certainty and more exploration possible, with many possible destinations at the end.

Finally, when you’re data aware, you are thinking more broadly. You understand the landscape of transportation, and are aware of the many opportunities in the transportation space (Figure 1-7). There are many options, many different timeframes, and many different ways of getting somewhere. In fact, there are even many places you could choose to go. You’re thinking about many kinds of problems with all kinds of data available to you. You’re engaging with many types of data and many methods. (This is a philosophy about data.)

Figure 1-7. Mapping, transportation, and navigation are a good way to think about being data aware: trains are just one method of getting around the landscape, and there is a whole world available to explore.

Fundamentally, the difference between data-aware and instinct-driven design comes down to what you rely on to inform your design decisions. With data-aware design, data is a creative production process and is the primary decision-making tool when and only when the data itself has been well designed and has been proven to be what we call fit for purpose. With instinct- or experience-driven design, decision making is more experimental.

Both paths can lead to great design. Is one way right? Absolutely not. Are these methods mutually exclusive? No. For us, the “right” approach to design will vary depending on the nature of the problem you are trying to solve and how you operate best; it will almost always require a balance between leveraging experience, instinct, and data. You need to have great instinct and experience to be a great designer regardless. Relying on data to help augment your existing skills adds one more great tool to your toolkit.

What Does This Mean for You as a Designer?

User-centered design and data applied to understanding behavior are both focused on establishing effective, rewarding, and replicable user experiences for the intended and current user base of a business or product. We believe that data capture, management, and analysis is the best way to bridge between design, user experience, and business relevance. Based on our previous descriptions, this is why we believe a data-aware approach to designing great user experiences is a better description of what we aspire to rather than the more commonly used data-driven terminology, as we believe data feeds into a creative design process, and is itself varied and creative, providing many possible ways to approach a problem.

But first, let’s review some of the assumptions we are making about which kind of designer you are. Given you picked up this book, we assume that:

• You’re interested in crafting great user experiences, and have some goals that involve changing or influencing the behavior of the users of your product or service.

• You are curious about human behavior as it relates to your product, and you are already making observations of users and how they use your product, even if only informally.

• You are thinking carefully about who your current users are, and who might become your users in the future.

• You are trying to solve many different kinds of problems and determine what works best for your users.

• While you may not have a background in statistics or be a data scientist, you are interested in becoming familiar with how you could get involved with the design of experiments to test out your ideas.

We believe that experimental methods and the data they yield can help hone your craft, improve your products, and concretely measure your impact on users and ultimately on the business. In sum, we believe that becoming familiar with the ways in which experiments carried out on the internet with large numbers of users where you can gather large amounts of disparate data types–that is, experiments at scale—can help you in your design practice. We believe that great design and smart data practices are key to strategic impact in any business.

Data Can Help to Align Design with Business

In our view, being data aware is also a good foundation for cross-functional collaboration within your business, whether large, medium, or small. It’s an excellent way to have impact upon and create alignment between design and business goals, focusing on the critical part of any business: providing the best possible service to your customers and clients, understanding their goals and concerns, and addressing their frustrations. Being user focused and data aware means you and the people you work with should also be actively contributing to the creation of meaningful business goals that are focused on the greatest asset of any business: your users.

What this means is that designing the user experience should also involve sketching out what data you will need that will help you understand your designs. Design the data capture, analysis, and questions as part of your design process. Be clear about the data that will best help you measure and articulate the effect of your design on your users, and therefore your business. Being smart about data in your decision making has considerable advantages. First, having common success metrics within your company can help designers and the broader product team to align around common goals, understand your target users, and understand the desired behaviors for business growth and maintenance. Data helps you, your team, and your organization make decisions that are based on evidence. Data can help you counteract questionable or poor decisions that may be reflections of the assumptions and beliefs of those in positions of power in the room. In addition, data can help you build empathy with your users; it can give your users a voice at the decision-making table with you as their advocate.

Second, data helps articulate the potential impact of design in service of meeting those goals. Data provides another means for you to defend your design decisions and root them in the needs of your users and your business. It also provides a concrete measure of the value that design brings to your business. Whether or not you believe that you should have to defend the value of your discipline, we find that showing how design has impacted the bottom line can help communicate to and educate stakeholders on why design is so important.

Finally, we believe that leveraging data will also help you become a better designer, as you will get to test your assumptions and hone your instincts with evidence. Your existing design process is likely already based on data—things you’ve observed about people and the world. As you continue to incorporate data (and more sophisticated forms of data), you’ll sharpen your understanding of how people behave, and therefore how to design the best experiences for them.

Notably, the actual data that reflect the users’ journey is sometimes not accessible to us. However, it is intuitively obvious that, as you get to know your users better by looking at what they do and how they react to your designed journey for them, you’ll become more adept at understanding the different kinds of experiences that engage them and that don’t. Data that you capture around your designs will help you reflect on whether your design achieved the goals you set out for it. As a simple example, let’s say you are designing a new sign-up flow. You can use data to measure if the design changes you made resulted in more people signing up or if you see them dropping out of sign-up flow at different points; this can also help you understand where your design might be confusing. Using data, you’ll get better at understanding which kinds of things will be more or less impactful on them and will therefore show or not show in the resulting metrics. We’ll talk more about this in later chapters. After all, one of the core reasons to take a data-aware approach to design is to be able to engage in an ongoing conversation with your customers through the data so that you can create better experiences for them.

On Data Quality

We want to encourage you to inform your design decisions by specific, objective evidence: data. Using data in your decision making entails that you reflect on the quality of the data, on what data is right for the decision-making setting—that you critically engage with the following:

• Question relevance (Are we asking the right questions?)

• Data appropriateness (Does it answer our questions?)

• Data quality (Is the data reliable? Did we lose something in data collection/curation? Did we bias the data?)

It also requires that we ask:

• Would different data and/or a different analysis be more appropriate? Are we doing what is convenient rather than what is right?

Jon Wiley, Director of Immersive Design at Google, has been working with data as a designer for over 10 years on some of the most popular and most used experiences in the world. At Google, he leads the user experience for all things virtual reality (VR) and augmented reality (AR). We spoke to Jon when he was leading the Search UX team about his experiences with A/B testing at a company known for its world-class use of data in the design and decision-making process.

Here’s his take on the topic:

Design is not art. Design does not exist for design’s sake. Design is about problem solving. There is a little bit of backlash sometimes among designers toward metrics and data. Maybe they feel like it’s taking some of their creativity away or they don’t want to be a slave to the numbers, that sort of thing.

I think designers have a responsibility to know whether or not they’ve actually solved a problem. Now, there are multiple ways of doing that. You can go out and just talk to users. You can observe them. If you can inject a bit of rigor into it, significant measurements, then you can decide ahead of time what success looks like for a problem and evaluate [the design] against those measures. One of the things that’s wonderful about working at Google is that we actually have so many really smart computer scientists and statisticians and an amazing ability to measure.

Let’s say I had a design with a blue button and everybody said it was great. Then I had another design that was a green button, and everybody said the same thing. If I go and run an experiment comparing the two, I can say, “Well, you know what. I can tell you, with statistical significance, this blue button is actually better and people will complete this task faster with the blue button.”

If you have that level of data, of course, you’ll use it. On the other hand, you don’t want to be ruled by the numbers. One of the things we discovered is that increasing the number of things that you measure or improving the fidelity of your measurements often actually doesn’t result in certainty. It doesn’t actually result in something that is crisp as this one is better than that. It just reveals a deeper complexity that there are actually more things involved. Then it really becomes a balance. We still have to have an intuition. We still have to make a judgment about what’s important and what isn’t.

It’s still down to the human behind the actual product. Metrics, data, and A/B testing are tools. They’re very important tools. I feel lucky to be able to say, “I know this is solving a user’s problem.” But these tools are not the sole mechanisms by which we make a judgment.

John Ciancutti from the startup 60dB also has some great thoughts about the relationship between data and design that we also wanted to share here. John has a wealth of knowledge about data-driven product development. He’s built his career at some of the industry’s leaders in leveraging data, including Netflix, Facebook, and Coursera. Now he’s applying his well-honed expertise to his startup, 60dB, which brings personalized audio news and stories directly to a user’s phone. John shared some thoughts from his time as Chief Product Officer at Coursera about the challenges of getting started with using data as a designer. He believes that the data and design disciplines have been traditionally siloed in how they’re taught and practiced. He shares how education and developing an intuition for data is a key step in getting started designing with data:

At Coursera, we hired a very promising designer out of school. She had this foreign relationship, an otherness with respect to data and data analysis. It’s just not a part of the work that she did in school, or as she was building her portfolio. The tension is natural because it’s like, “I don’t understand, it’s foreign, I’m not good at it.” Or, “it’s influencing my process, which I don’t want. I like the process I have.”

There are fundamentals to using data, like understanding the basics of statistics. I don’t think it’s intuitive that you could go ask a hundred other people about a question and begin to infer about thousands more. That’s not intuitive, because you think, “I’m not those people. They don’t know me, and I don’t know them.” Unless you really understand how statistics works and have done the math and seen cases where it’s worked, you’re not going to trust these random people as a source of truth.

Until data and the rudiments of analysis and data are part of the training of people who are trying to do design, it’s hard. As a designer, you are probably more capable than you recognize to raise some great points around data and so forth, but you just don’t know how to think about it yet because it’s not familiar.

We hope that this book will help you develop a familiarity with some of these key data concepts, and how they can be applied in a design setting.

With a Little Help from Your Friends...

As you start to incorporate the usage of data in your day-to-day work, you’ll find that in addition to navigating the data itself, you’ll also need to navigate the various important roles around data collection and get to know the people who work with data in different ways in your organization. In some cases, one person may carry out and represent each of the roles we describe—one user researcher, one data scientist, and so on. Or perhaps an entire team is dedicated to one particular role—many companies now have large teams of data scientists whose sole job is to carry out experimental tests and conduct data analysis. Or, in other instances, you will find one person playing multiple roles conducting usability tests, managing large data sets and conducting analyses, and so on. Many startup companies have one person who fulfills multiple roles simply because of their small size. However, while these roles may be fulfilled differently, there are two key functions. One is capturing, managing, summarizing, and analyzing the data to make it interpretable; the second is analyzing and using the data in multiple ways to reveal insights and generate business relevant information. We call those who are involved in managing and capturing data “producers,” and those who rely on data to inform their thinking “consumers.” A single individual might play both roles.

As you might have guessed, we believe that the practice of integrating design and data should be highly collaborative. You should actively work to leverage the knowledge and skills of people from the other disciplines and backgrounds that we outline here. Having a diverse range of opinions and inputs is always essential to any creative process. And taking a data-aware approach to design doesn’t mean that the full burden of gathering and analyzing the data needs to be on your shoulders alone; rather, you (and anyone else you work with) will each bring your own discipline-specific skillsets that can together create stronger and more thoughtful questions, and better approaches to answering those questions.

We’ve recently noticed that many companies are beginning to build closer organizational relationships between their design team and many of the folks working on and thinking about data—whether that means using open floor plans to allow for more spontaneous collaboration, or shifting around org charts. However, even if your organization hasn’t yet started making these kinds of changes, we encourage you to build your own relationships. Your “data friends,” as we’ll call them, can be invaluable resources for both designers who have never worked with data before and designers who have become experts at using data. We’ll devote much of Chapter 7 to discussing these organizational and cultural topics.

Data Producers

The people we often find associated with the generative side of data are data analysts, data scientists, user researchers, designers, and marketers. Let’s break them out a little.

Data analysts and scientists² should be involved throughout the lifecycle of a product. They take the large amounts of data collected from a product and then help to clean, interpret, transform, model, and validate that data. All of this work is done with the intent of helping the business to make better decisions. Data analysts and scientists often bring insight that can help the business predict where it needs to go, but they can also help to analyze the resulting data from a business decision to understand if the business accomplished what it set out to do. Data scientists may also be driving the discussion around defining metrics and measurements for your business and users. Common backgrounds for data analysts and scientists will include statistics, information management, computer science, and business intelligence.

Your friends in analytics know a great deal about your users through rich, generally large-scale data about how your users actually interact with your product. Because they have access to the logging data for your product, they can give you an idea of how past design and product changes have introduced metrics your company measures, and tell you confidently about how many users are currently using different features of your product, and at what frequency. For instance, if you’re thinking about removing a particular feature, you might ask your analytics friends about how many users are using that feature, and whether that feature use has historically had any relationship to whether your customers continue to use your product—that is, whether the feature has historically correlated with usage. Generally, your analytics friends are also responsible for analyzing A/B test results. You’ll find that they have great historic knowledge of how various ideas have performed in the past and can provide you with a lot of helpful guidance on structuring A/B tests.

User researchers³ are complimentary to data analysts and scientists and in some cases will overlap with them in terms of skills and interests—especially as user researchers start to do more of their work on a bigger and bigger scale. Typically, user researchers champion the user by seeking to understand who your users are and what their needs are. They are interested in both attitudinal and behavioral information about your users. They may focus on qualitative information gathering via interviews, surveys, diary studies, and other forms of ethnographic research; however, many user researchers also work with quantitative forms of data as well. Common backgrounds for people involved in user research might include social and behavioral sciences like psychology, cognitive science, sociology, or perhaps anthropology, as well as the backgrounds that many designers have. User researchers and analysts should also be consumers of each other’s data. Such sharing allows a broader understanding of user behavior with the product or service to develop; it helps to alleviate myopic and overly focused, feature-specific insights.

If you don’t know anything about a particular domain or about your users in general, you could work with your user research friends to understand who your users are, their needs and desires, and the aspects of your experience that are most frustrating to them right now. Because their job involves so much time observing the types of tasks that users struggle with, understanding the contexts in which they live, and asking them about their needs, user researchers have a keenly honed intuition about your users. Because many user researchers are trained in mixed methodologies—that is, multiple methods that can be combined to give faceted insights—they can also help you figure out what type of data is most suited to your particular questions.

Designers, we believe, need to also be concerned with the generation of data. It’s not new for designers to seek information about our users or to gather information about our designs as we evaluate them. In some companies that don’t have the room for dedicated user researchers or analysts, the designer will have to step into those roles occasionally to get the information that they need to generate their designs. Designers can also play a key role in the generation of data based on what they design and choose to prototype. Designers can reflect on existing business metrics, as well as think about any additional metrics with respect to usability or the user interface. Of course, designers may also be actively involved in understanding the data that comes back from user researchers or analysts on their own design throughout the product development cycle. The key for designers is to interpret these results and understand them within the context of the larger business and product.

Marketers can also be great collaborators in terms of data generation. Too often we find that a weak tie between product and marketing means that a lot of valuable information that could be shared by both teams can get lost. Members of the marketing team seek to understand the target audience and the market size of that audience, and as a result often generate a lot of data around customers. Many people in marketing will have a strong business background.

Marketing is probably the place to check if you want an expert opinion about your users’ demographics and target audiences. In Chapter 2, we will talk about the importance of being aware of the differences between your existing users and new users. Your marketing team will be able to help you understand what differences might exist between different types of users and important behavioral patterns based on age, gender, geolocation, culture, language, and other important features. The marketing team is also often responsible for understanding the competitive landscape and helping you understand what other products are in your business space. Having an understanding of how your competitors are faring and how their approach to users might be similar or different from yours can often help to give you ideas for how you might adjust or change your own user experience.

Data Consumers

On the consumption side of data roles, we find people who are actively taking the insights generated from the data to help them make decisions about how to push the business forward. In this bucket, we typically find business managers, product managers, and, of course, designers.

Business managers and product managers look to data to get stronger insights into how the business is performing. Business metrics are monitored and used to provide a health check on how the business is performing. They also look for impacts of business decisions and to see if the changes that are being done in the product are performing as expected. Formulating questions, designing experiments, and analyzing data to address issues that make sense from a design perspective as well as a business perspective can create alignment with business partners, collaborators, and company leadership about the target audience(s) and the desired behaviors for business growth and maintenance. Having a clear picture of what matters to your business will help you to plan and structure your quantitative tests and qualitative engagements.

If you are working with a product manager, they will be a critical partner in helping to set the context for your overarching business goals and the metrics you’ll be using to measure your success (or failure) by. You’ll be checking in with them periodically throughout the process to help generate ideas, but also to make sure that your designs and the “experiments” that you are conducting are reflective of the things you want to learn from your customers and in alignment with your business goals.

Engineers can be engaged along the way to ensure that as your designs are being built, you are also building in ways to track and capture the data that you are interested in. You want to be sure that the data you gather at the end of the process provides you with a complete picture of how your designs are impacting your users. You’ll need to have the tools in place to measure all the different pieces of data you might want to capture about your users, and about how they interact with your product, and this is something you’ll partner with engineers on. We’ll talk more about some of these considerations in Chapter 6.

The roles we have described here may not have strict boundaries or definitions. In small companies, for example, you will find people who take on several of these roles. However, as you might find yourself playing into either a consumer or producer of data it can be helpful to understand which side of this divide it is that you are fitting into at that specific point in time.

Of course, there could be many other people in your organization who might use data. Our list here is just a starting point, and we encourage you to keep your eye out for other people with whom you could collaborate.

What If You Don’t Have Data Friends (Yet)?

We just introduced many roles to you, and how those who fill these roles may work with data. Be thoughtful about who you’re already associated with organizationally or perhaps are simply friends with. Also seek to form stronger bonds with those you don’t know yet. We encourage you to reach out to those people to understand how they’re already working, and learn how your own work might fit alongside or complement what they do. Think about how you may work together to set a data-aware agenda across your organization.

We also recognize that many of you might work in smaller organizations that don’t have dedicated user researchers, analysts, or marketing teams just yet. However, this doesn’t mean you can’t start learning about your user! Many internet companies have started offering affordable services to help you begin building expertise about your users. For instance, UserTesting.com records real users interacting with your site or app, and provides insights about the users you’re interested in learning about. Optimizely provides tools that allow you to A/B test your web and mobile experiences. There are also a number of companies, such as Qualtrics and SurveyMonkey, that help you to run surveys and gather other kinds of data from your users. These types of services can supplement your in-house teams with external data experts, helping you get started learning. For a list of some companies that you can utilize, see the “Resources” appendix.

Themes You’ll See in This Book

We would like to highlight just a few of the themes that you’ll be seeing throughout this book (we may not reflect these in every chapter, but you will see them crop up in at least a few of them):

• First and foremost, working with data is a creative process. Your design background and creativity can help you to enhance experimentation and data practices in your company. Diving into experimentation and data may well also improve your design practice, as you develop your understanding of how the two areas can work in synergy to give your users the best possible experiences.

• You need to understand the problem you’re trying to solve, the users you’re solving for, and the business you’re in. The data you collect, and how you choose to collect it, should be mindful of these factors.

You’ve already collected many tools in your toolkit that are essential for designing with data:

• Design intuition isn’t lost as a consequence of leveraging data, and in fact, is important to shape the questions that data can answer and how to answer them.

• Triangulation between multiple sources of data tells a more complete picture of your users, and helps you reduce the risk of being misled by data.

• You will need to work with others to be successful. You don’t need to be an expert in data in addition to design. Rather, you should approach this book and designing with data with the goal of understanding how your skillset as a designer can help push the process of collecting data forward. Other folks in your organization will bring complementary skillsets, which combined, can plan and execute the best data strategy.

Summary

We believe you’ll find that your training as a designer will give you a lot of advantages when working with data. Designers are often natural facilitators and you’ll find that you can be fairly adept at driving participation from many different parts of your organization and from many people with different kinds of skills as a result. Because designers are usually comfortable representing the user and being especially empathetic to user needs, you’ll find that incorporating the language of data along the way, and working with others to do so, will amplify your ability to speak on behalf of the user and to understand what ultimately works best for them.

There isn’t a “one size fits all” approach to data and design, and understanding the nuances of data driven versus data informed versus data aware can be a powerful tool in design. Depending on the types of problems you are trying to solve, and how far along you are in your design process and product maturity, you may find yourself leveraging data in different ways. We encourage you to always be open minded about the other approaches you could be taking to solve your problem, and how you might use data in different ways if your approach was different.

We are focused on giving you a basic understanding of the relationship between data, business, and design, rather than teaching you how to design or making you an expert statistician or data scientist. Though formal and systematic incorporation of data from large-scale experiments into design is relatively recent, we believe this is the beginning of an exciting and long era to come. We believe that data and design are tools that you use to build great experiences for your users. And, if you are building great experiences for your users, then you have a great foundation for your business. We will feel successful if this book convinces you that understanding the basics of experimental methodologies and the data that you can gather about your users from applying such methods will make your design(s) and therefore your business better, and if you feel that you are not just willing but keen to engage with data in your organization and beyond.

We hope this chapter has given you a foundation for the approach that we would like to encourage as you start to work with data in your design decisions. As a designer, you will play both the role of a producer and a consumer of data at various times. We want you to feel empowered by data and to recognize that by adapting this framework, you’re actually engaging in a two-way conversation with your users where both data and design can be a useful tool in that communication.

Questions to Ask Yourself

• What kinds of data does your company use?

• In what ways is your company data driven? Data informed? Data aware?

• Are there large-scale experiments being run in your company already?

• Who is responsible for those experiments?

• Who is currently producing and consuming data in your company?

• Who do you currently work with that can help you on your quest to integrate data into your design process?

• What kind of people might you want to have access to that you currently don’t (e.g., data scientists, analysts, etc.)? What external or other resources could you use instead?

• How would you naturally fit in or adjust the way you work with “data friends” or other functions in your company to support the integration of data and design?

• What is the designer’s role in producing data? What areas for improvement do you see in the relationship between data and design?

• Are there data “gaps”? How could you close those gaps?

• What kinds of questions would you like to ask about your users? What answers will help you in your work?

¹ http://andrewchen.co/2012/05/29/know-the-difference-between-data-informed-and-versus-data-driven/

² “Data science” is recent. It was in the 1960s that statisticians like John Tukey started to think more about what it was to bring a scientific approach to data analysis. In the 1970s we see these analysts start to recognize that the power they can bring to the data is to fill it with insight and more information. As we jump ahead to today, there is no question that data science is a term that now involves all the work that is done to capture, measure, and interpret the vast amount of data that represents our users on a daily basis. See Doing Data Science by Cathy O’Neil and Rachel Schutt (O’Reilly).

³ For more information, see “UI, UX: Who Does What? A Designer’s Guide to the Tech Industry” (http://www.fastcodesign.com/3032719/ui-ux-who-does-what-a-designers-guide-to-the-tech-industry).