3.11 Reference posterior distributions
3.11.1 The information provided by an experiment
Bernardo (1979) suggested another way of arriving at a reference standard for Bayesian theory. The starting point for this is that the log-likelihood ratio 
can be regarded as the information provided by the observation x for discrimination in favour of 
against 
(cf. Good, 1950, Section 6.1). This led Kullback and Leibler (1951) to define the mean information in such data to be
(cf. Kullback, 1968, and Barnett, 1999, Section 8.6).1 Note that although there is a relationship between information as defined here and Fisher’s information I as defined in Section 3.3 earlier (see Kullback, 1968, Chapter 2, Section 6), you are best advised to think of this as a quite separate notion. It has in common with Fisher’s information the property that it depends on the distribution of the data rather than on any particular value of it.
Following this, Lindley (1956) defined the expected amount 
of information that ...
Get Bayesian Statistics: An Introduction, 4th Edition now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
