A Primer on Information Theory 



23 



of F= 0.53 for/) = 0.37. Inspection of the graph reveals some more important 

 properties of the function F{p) : 



(5) nO) = 0: 



When a particular class of events is certain not to occur {p = 0), then it does 

 not contribute to the measure of uncertainty. 



(6) F(1) = 0: 



F(p) = - p logp p 



F(p) 0.3 



Fig. 1. Graph of F{p) as a function of/? 



When a particular class of events is certain to occur {p = 1), i.e. excludes 

 all other classes, then there is no uncertainty about the outcome. 



(7) Effect of Averaging: 



F 



> i[F(p,) + F(p^)] 



The function of the average is greater than or at least as large as the average 

 of the function. When the probabilities associated with two disjoint categories 

 are averaged, then the uncertainty becomes larger. Figure 2 is a graphical 

 demonstration of this effect. 



The extreme case of averaging occurs if all r categories in a classification 

 are considered equiprobable. Then, 



Pi') = 7 

 I, 1 1 11 



max. of H(.x) = — ^ - log., - = — /• • - lo?., - 

 ,-1 /• ^- r r - r 



max. of H(.x) = log, r 



