A Primer on Information Theory 



25 



10. Estimate the uncertainty of a motorist like the one described at the beginning of this 

 section. 



11. Certain languages have considerably fewer letters than English (that is, about 18 to 

 20), yet the information content per letter is nearly the same. How is this possible? 



12. A situation has an unlimited number of alternatives, with probabilities of 1/2, 1/4, 

 1/8, 1/16, etc. in geometric progression. What is the measure of uncertainty? 



F(P|)+ Flp^) 



FlPl + Pg) 



fj P,+ Pg P2 p,+ P2 

 2 



Fig. 3. Graphical demonstration of the effect of pooling 

 The function of the sum is on the intersection between the curve and the 

 ordinate over the sum; the sum of the functions is on the intersection of the same 

 ordinate with a straight line through the origin and the midpoint of the straight 

 line which connects the intersections of the curve with the ordinates over pi and 

 P2, hence: 



F(p, + P2) < F(p,) + F(p,) 



IV. INFORMATION MEASUREMENTS PERTAINING TO 

 TWO RELATED VARIABLES 



In the two preceding sections we have discussed how to represent information, 

 and how to measure amounts of information. Both procedures become impor- 

 tant if information is to be manipulated. The manipulation most commonly 

 used is communication. 



