22 Henry Quastler 



words, the uncertainty concerning independent events should be the sum of 

 the individual uncertainties. 



Let y be an event with categories j and associated probabilities p{j). Let 

 p{i,j) be the probability of the event pair that x falls into category / and v 

 into category y. Then, the function 



Hix,}') = -lp{i,j)\og2p(i,j) 



will measure the uncertainty associated with the event pair. 

 If X and y are independent events, then 



p{Uj)^p{i)-p{]) 



As a matter of fact, this relation is often used to define independence. In this 

 case, we have 



H{x, j) = - 2 p{i.j) logo p{i) ■ pij) 



i.j 



= -lp(hj)^og^pii) - lp('J)\oz.2p(j) 



It is known that 



J.piUj)=p{i) 



j 



IpiUj)=p(j) 



Substituting these expressions, we obtain 



Hix, >0 = - 2 Pii) log2 /XO - 2 p(j) loga p(j) 



i . 



= H(x) + H(y). ^ H^^^) ^ H^'^^^'i^f^^ 



Thus, the Shannon-Wiener function fulfills the postulate of additivity. 



(4) Natural Scale— X\yQ prototype of uncertainty is that associated with a 

 50-50 choice. So, the unit of uncertainty should be the uncertainty associated 

 with this situation. In this case, both/s have the value 1/2, and 



Hix) = -(1/2 log2 1/2 + 1/2 log2 1/2) - 1 



Thus, the Shannon-Wiener function is seen to have an appropriate scale factor. 



We have derived the infonnation function from the postulate of eflScient 

 binary representation, and have found that the function so defined has the 

 desirable properties of independence, continuity, additivity, and natural scale. 

 We could have started differently, setting up these four properties 2i^ postulates. 

 It can be shown that these four postulates (or other sets of four similar postu- 

 lates) define uniquely the Shannon-Wiener function. Working it this way, 

 we would have derived the fact that the function so defined has the desirable 

 property of efficient binary representation. 



The function F{p) is plotted against/; in Fig. 1. The graph shows a curve 

 which originates and terminates at F = 0, and has a flat top with a maximum 



