Lean Analytics

Table of contents

1. Praise for Lean Analytics
2. Dedication
3. Foreword
4. Preface
   1. Who This Book Is For
   2. How This Book Works
   3. The Building Blocks
      1. Customer Development
      2. Lean Startup
   4. We’d Like to Hear from You
   5. O’Reilly Safari
   6. Thanks and Acknowledgments
5. I. Stop Lying to Yourself
   1. 1. We’re All Liars
      1. The Lean Startup Movement
      2. Poking a Hole in Your Reality Distortion Field
      3. Airbnb Photography—Growth Within Growth
   2. 2. How to Keep Score
      1. What Makes a Good Metric?
      2. Qualitative Versus Quantitative Metrics
      3. Vanity Versus Real Metrics
      4. Eight Vanity Metrics to Watch Out For
      5. Exploratory Versus Reporting Metrics
      6. Circle of Moms Explores Its Way to Success
      7. Leading Versus Lagging Metrics
      8. Correlated Versus Causal Metrics
      9. Moving Targets
      10. HighScore House Defines an “Active User”
      11. Segments, Cohorts, A/B Testing, and Multivariate Analysis
          1. Segmentation
          2. Cohort Analysis
          3. A/B and Multivariate Testing
      12. The Lean Analytics Cycle
      13. Evaluating the Metrics You Track
   3. 3. Deciding What to Do with Your Life
      1. The Lean Canvas
      2. What Should You Work On?
      3. Create a Lean Canvas
   4. 4. Data-Driven Versus Data-Informed
      1. How to Think Like a Data Scientist
      2. Lean Startup and Big Vision
6. II. Finding the Right Metric for Right Now
   1. 5. Analytics Frameworks
      1. Dave McClure’s Pirate Metrics
      2. Eric Ries’s Engines of Growth
         1. Sticky Engine
         2. Virality Engine
         3. Paid Engine
      3. Ash Maurya’s Lean Canvas
      4. Sean Ellis’s Startup Growth Pyramid
      5. The Long Funnel
      6. The Lean Analytics Stages and Gates
   2. 6. The Discipline of One Metric That Matters
      1. Moz Tracks Fewer KPIs to Increase Focus
      2. Four Reasons to Use the One Metric That Matters
      3. Solare Focuses on a Few Key Metrics
      4. Drawing Lines in the Sand
      5. The Squeeze Toy
      6. Define Your OMTM
   3. 7. What Business Are You In?
      1. About Those People
      2. The Business Model Flipbook
         1. Lots to Choose From
      3. Six Business Models
      4. Pick Your Business Model
   4. 8. Model One: E-commerce
      1. What Mode of E-commerce Are You?
      2. A Practical Example
         1. Conversion Rate
         2. Purchases Per Year
         3. Shopping Cart Size
         4. Abandonment
         5. Cost of Customer Acquisition
         6. Revenue Per Customer
      3. WineExpress Increases Revenue by 41% Per Visitor
         1. Keywords and Search Terms
         2. Recommendation Acceptance Rate
         3. Virality
         4. Mailing List Click-Through Rates
      4. Offline and Online Combinations
         1. Shipping Time
         2. Stock Availability
      5. Visualizing the E-commerce Business
      6. Wrinkles: Traditional E-commerce Versus Subscription E-commerce
      7. Key Takeaways
   5. 9. Model Two: Software as a Service (SaaS)
      1. Backupify’s Customer Lifecycle Learning
      2. Measuring Engagement
      3. Churn
         1. Churn Complications
      4. Visualizing the SaaS Business
      5. ClearFit Abandons Monthly Subscriptions for 10x Growth
      6. Wrinkles: Freemium, Tiers, and Other Pricing Models
      7. Key Takeaways
   6. 10. Model Three: Free Mobile App
      1. Installation Volume
      2. Average Revenue Per User
      3. Percentage of Users Who Pay
      4. Churn
      5. Visualizing the Mobile App Business
      6. Wrinkles: In-App Monetization Versus Advertising
      7. Key Takeaways
   7. 11. Model Four: Media Site
      1. Audience and Churn
      2. Inventory
      3. Performance and the Sessions-to-Clicks Ratio
      4. Ad Rates
      5. Content/Advertising Trade-off
      6. Visualizing the Media Business
      7. Wrinkles: Hidden Affiliates, Background Noise, Ad Blockers, and Paywalls
      8. Key Takeaways
   8. 12. Model Five: User-Generated Content
      1. Visitor Engagement
      2. Content Creation and Interaction
      3. Engagement Funnel Changes
      4. Value of Created Content
      5. Content Sharing and Virality
      6. Notification Effectiveness
      7. Visualizing a UCG Business
      8. Wrinkles: Passive Content Creation
      9. Key Takeaways
   9. 13. Model Six: Two-Sided Marketplaces
      1. What DuProprio Watches
      2. Rate at Which You’re Adding Buyers and Sellers
      3. Rate of Inventory Growth
      4. Buyer Searches
      5. Conversion Rates and Segmentation
      6. Buyer and Seller Ratings
         1. Percent of Flagged Listings
      7. Visualizing a Two-Sided Marketplace
      8. Wrinkles: Chicken and Egg, Fraud, Keeping the Transaction, and Auctions
      9. Key Takeaways
   10. 14. What Stage Are You At?
       1. Pick the Stage That You’re At
   11. 15. Stage One: Empathy
       1. Metrics for the Empathy Stage
       2. This Is the Best Idea I’ve Ever Had! (or, How to Discover Problems Worth Solving)
       3. Finding a Problem to Fix (or, How to Validate a Problem)
       4. Signs You’ve Found a Problem Worth Tackling
       5. Running Lean and How to Conduct a Good Interview
       6. How to Avoid Leading the Witness
       7. Convergent and Divergent Problem Interviews
       8. How Do I Know If the Problem Is Really Painful Enough?
          1. Calculating the Scores
       9. Cloud9 IDE Interviews Existing Customers
       10. How Are People Solving the Problem Now?
       11. Are There Enough People Who Care About This Problem? (or, Understanding the Market)
       12. What Will It Take to Make Them Aware of the Problem?
       13. A “Day in the Life” of Your Customer
       14. Finding People to Talk To
       15. Getting Answers at Scale
       16. LikeBright “Mechanical Turks” Its Way into TechStars
       17. Creating an Answers-at-Scale Campaign
       18. Build It Before You Build It (or, How to Validate the Solution)
       19. Localmind Hacks Twitter
       20. Before You Launch the MVP
       21. Deciding What Goes into the MVP
       22. Measuring the MVP
           1. Don’t Ignore Qualitative Analytics
           2. Be Prepared to Kill Features
       23. Static Pixels Eliminates a Step in Its Order Process
       24. A Summary of the Empathy Stage
       25. Should You Move to the Next Stage?
   12. 16. Stage Two: Stickiness
       1. MVP Stickiness
       2. Iterating the MVP
       3. qidiq Changes How It Adds Users
       4. Premature Virality
       5. The Goal Is Retention
       6. Seven Questions to Ask Yourself Before Building a Feature
       7. How Rally Builds New Features with a Lean Approach
       8. How to Handle User Feedback
       9. The Minimum Viable Vision
       10. The Problem-Solution Canvas
       11. VNN Uses the Problem-Solution Canvas to Solve Business Problems
       12. A Summary of the Stickiness Stage
       13. Exercise #1: Should You Move to the Next Stage?
       14. Exercise #2: Have You Identified Your Biggest Problems?
   13. 17. Stage Three: Virality
       1. The Three Ways Things Spread
          1. Inherent Virality
          2. Artificial Virality
          3. Word-of-Mouth Virality
       2. Metrics for the Viral Phase
       3. Beyond the Viral Coefficient
       4. Timehop Experiments with Content Sharing to Achieve Virality
       5. Instrumenting the Viral Pattern
       6. Growth Hacking
          1. Attacking the Leading Indicator
          2. What Makes a Good Leading Indicator?
          3. Correlation Predicts Tomorrow
          4. Causality Hacks the Future
       7. A Summary of the Virality Stage
       8. Should You Move On to the Revenue Stage?
   14. 18. Stage Four: Revenue
       1. Metrics for the Revenue Stage
       2. The Penny Machine
          1. Penny Machines and Magic Numbers
       3. Finding Your Revenue Groove
          1. Where Does the Money Come From?
       4. Customer Lifetime Value > Customer Acquisition Cost
       5. Parse.ly and the Pivot to Revenue
       6. Market/Product Fit
          1. Review Your Old Assumptions
          2. Begin a Process of Elimination
          3. Deep Dive
          4. Find Similarities
       7. The Breakeven Lines in the Sand
          1. Breakeven on Variable Costs
          2. Time to Customer Breakeven
          3. EBITDA Breakeven
          4. Hibernation Breakeven
       8. Revenue Stage Summary
   15. 19. Stage Five: Scale
       1. The Hole in the Middle
       2. Metrics for the Scale Stage
       3. Is My Business Model Right?
       4. Buffer Goes from Stickiness to Scale (Through Revenue)
       5. The Three-Threes Model
       6. Finding Discipline as You Scale
       7. A Summary of the Scale Stage
   16. 20. Model + Stage Drives the Metric You Track
7. III. Lines in the Sand
   1. 21. Am I Good Enough?
      1. WP Engine Discovers the 2% Cancellation Rate
      2. Average Isn’t Good Enough
      3. What Is Good Enough?
      4. Growth Rate
         1. Is Growth at All Costs a Good Thing?
         2. Bottom Line
      5. Number of Engaged Visitors
         1. Bottom Line
      6. Pricing Metrics
      7. Socialight Discovers the Underlying Metrics of Pricing
         1. Bottom Line
      8. Cost of Customer Acquisition
         1. Bottom Line
      9. Virality
         1. Bottom Line
      10. Mailing List Effectiveness
          1. Bottom Line
      11. Uptime and Reliability
          1. Bottom Line
      12. Site Engagement
          1. Bottom Line
      13. Web Performance
          1. Bottom Line
      14. Make Your Own Lines in the Sand
   2. 22. E-commerce: Lines in the Sand
      1. Conversion Rate
         1. Bottom Line
      2. Shopping Cart Abandonment
         1. Bottom Line
      3. Search Effectiveness
         1. Bottom Line
   3. 23. SaaS: Lines in the Sand
      1. Paid Enrollment
         1. Bottom Line
      2. Freemium Versus Paid
      3. Upselling and Growing Revenue
         1. Bottom Line
      4. Churn
      5. OfficeDrop’s Key Metric: Paid Churn
         1. Bottom Line
   4. 24. Free Mobile App: Lines in the Sand
      1. Mobile Downloads
         1. Bottom Line
      2. Mobile Download Size
         1. Bottom Line
      3. Mobile Customer Acquisition Cost
         1. Bottom Line
      4. Sincerely Learns the Challenges of Mobile Customer Acquisition
      5. Application Launch Rate
         1. Bottom Line
      6. Percent Active Mobile Users/Players
         1. Bottom Line
      7. Percentage of Mobile Users Who Pay
         1. Bottom Line
      8. Average Revenue Per Daily Active User
         1. Bottom Line
      9. Monthly Average Revenue Per Mobile User
         1. Bottom Line
      10. Average Revenue Per Paying User
          1. Bottom Line
      11. Mobile App Ratings Click-Through
          1. Bottom Line
      12. Mobile Customer Lifetime Value
   5. 25. Media Site: Lines in the Sand
      1. Click-Through Rates
         1. Bottom Line
      2. Sessions-to-Clicks Ratio
         1. Bottom Line
      3. Referrers
         1. Bottom Line
      4. Engaged Time
         1. Bottom Line
      5. What Onsite Engagement Can Tell You About Goals and Behaviors
      6. Sharing with Others
         1. Bottom Line
      7. JFL Gags Cracks Up YouTube
   6. 26. User-Generated Content: Lines in the Sand
      1. Content Upload Success
         1. Bottom Line
      2. Time on Site Per Day
         1. Bottom Line
      3. Reddit Part 1—From Links to a Community
      4. Engagement Funnel Changes
         1. Bottom Line
      5. Reddit Part 2—There’s Gold in Those Users
      6. Spam and Bad Content
         1. Bottom Line
   7. 27. Two-Sided Marketplaces: Lines in the Sand
      1. Transaction Size
         1. Bottom Line
      2. What Etsy Watches
      3. Top 10 Lists
         1. Bottom Line
   8. 28. What to Do When You Don’t Have a Baseline
8. IV. Putting Lean Analytics to Work
   1. 29. Selling into Enterprise Markets
      1. Why Are Enterprise Customers Different?
         1. Big Ticket, High Touch
         2. Formality
      2. Legacy Products
         1. Incumbents
         2. Slower Cycle Time
         3. Rationality (and Lack of Imagination)
      3. The Enterprise Startup Lifecycle
         1. Inspiration
         2. Empathy: Consulting and Segmentation
      4. How Coradiant Found a Market
         1. Stickiness: Standardization and Integration
         2. Virality: Word of Mouth, Referrals, and References
         3. Revenue: Direct Sales and Support
         4. Scale: Channel Sales, Efficiencies, and Ecosystems
      5. So What Metrics Matter?
         1. Ease of Customer Engagement and Feedback
         2. Pipeline for Initial Releases, Betas, and Proof-of-Concept Trials
         3. Stickiness and Usability
         4. Integration Costs
         5. User Engagement
         6. Disentanglement
         7. Support Costs
         8. User Groups and Feedback
         9. Pitch Success
         10. Barriers to Exit
      6. The Bottom Line: Startups Are Startups
   2. 30. Lean from Within: Intrapreneurs
      1. Span of Control and the Railroads
      2. Skunk Works for Intrapreneurs
      3. Changing—or Innovating to Resist Change?
      4. Stars, Dogs, Cows, and Question Marks
      5. Swiffer Gives Up on Chemistry
      6. Doritos Chooses a Flavor
      7. Working with an Executive Sponsor
      8. EMI Embraces Data to Understand Its Customers
      9. The Stages of Intrapreneur Lean Analytics
         1. Beforehand: Get Buy-in
         2. Empathy: Find Problems, Don’t Test Demand
         3. Skip the Business Case, Do the Analytics
         4. Stickiness: Know Your Real Minimum
         5. Viral from the Start
         6. Revenue Within the Ecosystem
         7. Scale and the Handoff
   3. 31. Conclusion: Beyond Startups
      1. How to Instill a Culture of Data in Your Company
         1. Start Small, Pick One Thing, and Show Value
         2. Make Sure Goals Are Clearly Understood
         3. Get Executive Buy-in
         4. Make Things Simple to Digest
         5. Ensure Transparency
         6. Don’t Eliminate Your Gut
         7. Ask Good Questions
9. A. References and Further Reading
10. B. About the Authors
11. C. The Lean Series
12. Index
13. About the Authors
14. Copyright