Product Research Rules

Chapter 2. Everyone Is Biased, Including You

In 2016, product leader Hope Gurion (now of Fearless Product) took a job at Beachbody, a multilevel marketing company with a network of over 250,000 fitness coaches. The company was preparing for its annual coaching summit, which would bring 50,000 people together in a stadium in Nashville, Tennessee. It would be a great opportunity for the product team to interact with coaches. Hope and her team were responsible for managing an app called Coach Office, which the coaches used to organize their administrative work.

Up until then, Beachbody’s research on this app had focused solely on its top coaches: a high-performing 1%, or about 2,500 people, who had been with the company for a long time and were expert users. They had learned their way around the product and had no issues navigating around it and getting things done. This led the internal Beachbody stakeholders, who had come to know the top coaches over the years, to believe that everything was fine with the app. But what about the other 247,500 coaches—were they using the app well?

Further, the app, which had been developed by a third party, had not been built with smartphones in mind. (Remember, it was 2016!) To make matters worse, it wasn’t instrumented for usage analytics, either. The internal team had no data to help them understand app usage. So all they had to go on were the experiences and comments of these top coaches. They were biased toward one small segment of their users—and ...