24 



Henry Quastler 



In particular in a binary classification, 



r = 2 



max. of H{x) = 1 



Thus, the maximum uncertainty associated with two alternatives is one bit; it 

 occurs if both alternatives are equally probable (this is the case of the unbiassed 

 coin!). 



(8) Ejfect of Pooling: 



F(pi + P2X F(Pi) + np2) 

 The function of the sum is smaller than the sum of the functions. That is, 

 pooling of two classes in one equivalence class reduces uncertainty (exactly 



P|+ Po 



F(P|1 + F(P2J 



Fig. 2. Graphical demonstration of the effect of averaging 



by that uncertainty which is associated with the distinction between the two 



pooled classes). Extreme pooling results in a single category with probability 1 ; 



this means uncertainty 0. Figure 3 demonstrates the effect of poohng. 



The function F(p) = —p logg p has been tabulated. The reader is advised 



to use Fig. 1 to obtain approximate values for use in working the exercises 



below. For more precise values, one of the existing tables may be consulted 



(10, 11). 



EXERCISES 



7. Compute the uncertainty associated with: 



p(A) = .60 

 /•(non-A) = .40 



8. Compute H(x) for two alternatives, and plot the value against /7(A). 



9. Answer the question posed previously: which uncertainty is greater, that associated 

 with a situation (x) with three equiprobable alternatives, or that (y) where there are 4 possibili- 

 ties with probabilities .8, .1, .05 and .05. 



