A Primer on Information Theory 47 



10. A realistic description of his uncertainty might be: 



prob (55-64) = .95 



prob (55-54) -- .02 



prob (65-70) = .02 



prob (any other speed) = .01 



Within each range, all speeds are considered equiprobable. 

 We will derive the answer in two steps, obtaining first the uncertainty as to the speed range: 



Range p —p log., p 



.36 bits 



Next, we observe that the range from 55 to 64 miles per hour contains ten speeds (deter- 

 mined to the nearest mile) which are equiprobable. The uncertainty measure for ten equi- 

 probable categories has been found to be log., 10 = 3.32. This uncertainty will arise 95 times 

 out of 100; its expected contribution to the total uncertainty is 3.32 ■ 0.95 = 3.15. The other 

 ranges are treated equally : 



We thus need (on average) .36 bits to determine the range of speeds, and an additional 3.31 

 bits (on average) to identify the speed to the nearest mile, within the range. The total uncer- 

 tainty is 0.36 + 3.31 = 3.67 bits. 



Of course, different expectations would yield different uncertainties. 



1 1 . The letters occur with more nearly equal frequencies. 



12. Two bits. 



„, . , /323 323 104 104\ ^^ ^. 



13. i/(shape) = - — log, h — lo", — -- .80 bits 



\427 ^-427 427 "■427/ 



rrr , ^ /315, 315 112, 112\ ^. ,. 

 //(color) = - — loga h — log., — = .83 bits 



\427 ^427 427 ^-427/ 



17/1 K ^ ^96 , 296 27 , 27 19 , 19 

 //(color, shape) = - — log, 1 log., 1 log., — 



\ 427 ^- 427 427 ^' 427 427 ^" 427 



+ ^log„^) =1.26 bits 



427 ^- 427/ 



r(color; shape) = .80 -I .83 - 1.26 = .39 bits 



