Prediction Revisited

4Relevance

In Chapter 2, we reconsidered the distribution of individual attributes through the lens of pairs, and we used this lens to illuminate the connection between probability and informativeness. In Chapter 3, we reconsidered the association between pairs of attributes within the context of informativeness. Now we show how to combine informativeness with similarity to determine the relevance of observations. In addition, we reveal how relevance is related to linear regression analysis and how this relationship enables us to improve predictions.

Relevance Conceptually

By now we have learned what is typical for the attributes in our dataset: how much they vary and the extent to which pairs tend to vary together. Most importantly, we have become more knowledgeable historians, able to pinpoint the notable events in each attribute's history. The next step is to consider circumstances more holistically. By circumstances, we mean patterns that occur across many attributes at the same time. Put differently, circumstances are the collection of values that we choose to define an observation. They are more colorful, more descriptive, and more informative than any one attribute on its own. What if we can refine our historical understanding to identify the circumstances that truly matter: those that inform us about outcomes yet to be observed? To do this properly, we need all the richness of understanding we have developed up to this point.

Informativeness

Remember that information ...