Profit Driven Business Analytics

CHAPTER 6Profit-Driven Model Evaluation and Implementation

INTRODUCTION

As an alternative to adopting profit-driven analytical techniques, as discussed in the previous chapter, we may implement a profit-driven analytics strategy by looking at the profitability of analytical models in the evaluation step. This omits the use of complex modeling techniques and may therefore be of preference. We do want to warn the reader, however, that the process of developing and applying an appropriate and accurate profit-sensitive evaluation may turn out to be challenging. In this chapter, we support the development of such evaluation by discussing a series of advanced measures for profit-driven performance assessment of classification and regression models. We aim to provide deeper insight in the basic principles underlying these measures, and from these discussions we will as well arrive at some key guidelines regarding the operational implementation of these measures.

In the first part of this chapter, we will focus on classification models. We will start by discussing the most straightforward and intuitive profit-based evaluation approach, i.e., calculating the average misclassification cost. Related to this is the selection of a classification cutoff score, which can be tuned using the average misclassification cost to optimize the resulting profitability of the model. Consequently, the evaluation of a classification model has practical and important implications towards the implementation ...