Operationalizing the Data Lake

Chapter 11. Case Study: Ibotta Builds a Cost-Efficient, Self-Service Data Lake

Ibotta is a mobile technology company, founded in 2011, that is transforming the traditional rebates industry. They provide in-app cashback rewards on receipts and online purchases for groceries, electronics, clothing, gifts, supplies, restaurant dining, and more for anyone with a smartphone.

Today, Ibotta is one of the most used shopping apps in the United States, driving more than $7 billion in purchases per year to companies like Target, Costco, and Walmart. Ibotta has more than 27 million total downloads and has paid out more than $500 million to users since its founding in 2012. Maintaining a competitive edge in the ecommerce and retail industry is extremely difficult because it requires creating engaging and unique shopping experiences for consumers.

Prior to moving to a big data platform with Qubole, Ibotta’s data and analytics infrastructure was based on a cloud data warehouse that was static and rigid. This worked as long as the datasets were well structured and in tabular format. However, as the business grew, new and more complex data formats were being developed and ingested.

At the same time, Ibotta was heavily investing in new data analytics teams such as data engineering, decision science, and machine learning. The teams needed access to the same data, but each team needed a different way of interacting with the data. Data engineering needed a set of tools that allowed it to perform ETL ...