24 Information Theory
Introduction
In this chapter, the ideas associated with information and entropy are considered and related to probability defined in terms of incomplete knowledge. Mutual and self information are introduced, and these concepts are applied to various models for communications channels.
The concepts of information and related quantity entropy date back to the early days of thermodynamics. The concept of entropy arose in considerations connected with the theory of heat, and its relationship to information was not at first realized. However, in retrospect it is not surprising that the analytical treatment of information should have its origin in this area of physical science, for the subject of thermodynamics ...
Get The Cable and Telecommunications Professionals' Reference, 3rd Edition now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
