28 Henry Quastler 



Two-part Systems in General 



We proceed to a general treatment of a two-part system x, y. Let / and 7 be 

 the categories of x and v, respectively, and p{i) and p{j) the associated 

 probabilities. Further, let p{i,j) be the probability of the joint occurrence 

 [{x = i) and (y =;)]. 

 Then: 



H(x)^ -2 p{i) 10^2 p{i) 



i 



H(y) = -Ipij)^og,p(j) 



H(x, y) = - 2 p(i, j) logs p{i, j) 



ij 



We introduce the conditional probabilities, 



Piij) Prob { V = y if X = /} 



/>,.(O....Prob{x = /ifj=y} 



When X = i then y must have some value j with certainty (or probability 

 1.0), that is 



IPiiJ) = 1 

 j 



Equally, 



Ip^iO = 1 



i 



Furthermore, the probability of the joint occurrence [x ~ i and y = j] can be 

 factored into the product of the probability that x equals /, times the conditional 

 probability that y = j ii x — i; equally, it can be factored into the product of 

 pij) times Pj{i). So : 



P(i,j)=pii)-Pi{j) 



^Pij)-Pj(0 



The conditional probabilities yield naturally conditional uncertainties. For 

 instance, the uncertainty of j, if it is known that x = i, will be 



Hiiy) = -IPiij) loga/^XO 

 3 



The average uncertainty of j, under the condition that x is known, is designated 



by H/y). It is obtained as the weighted average of the //Xv)'s- 



i 



Substituting the value of H^{y), we get 



tJxiy) = -Ipii) 1 Piij) ^og2Pi(j) 

 I j 



and remembering that 



Pii}) - -jay 



we get 



