A Primer on Information Theory 



15 



each category, /, is represented by a code word of approximately —\og2pO) 

 digits; accordingly, its contribution to the weighted average is not far from 

 the ideal value of —p{i) log, /?(/), and the mean code length is only very slightly 

 greater than the limiting value of —]£/?(/) logg /?(/)• 



We have already met a situation where a binary code was less than optimally 

 efficient (in the sense of minimum length of code words); that was the case 

 of r equiprobable categories, when r was not an integral power of 2. In this 



