Information Processing Theory 45 



are undefined, being like, perhaps, phonemes, but not necessarily 

 so identified. It is only necessary to assume that words are com- 

 posed of these units and that the cost of a word is equal to the 

 sum of the costs of the units. To illustrate, take the special case 

 where each unit has the same cost attached to its use. The least 

 expensive words are tlien the ones composed of single units. If 

 there are M units, there are M such minimum cost words, M~ 

 double unit words (second in cost), and so on. 



The rank of a word will be determined by the number of words 

 which can be coded with Cr or fewer symbols. For example, if 

 words are coded as binary sequences, M = 2, and there are 

 fourteen codes of three or fewer digits (0, 1, 00, 01, 10, 11, 000, 

 001, 010, Oil, 100, 101, 110, ill). Thus a word of cost 3 will have 

 rank Hand )•{?>) = 14. 



In general, 



c c 



r{Cr) =i:ii/^=i:ii/^- 1 



1 - 71/ 

 1 - M 



so that 



M - 1 



71/^' - 1 = /• (71/ - l)iW-' 



Now, 



Cr = Cr log,/ 71/ 



= log,; [(71/'''- 1) + 1] 



= log,/ [(7I/'''- - 1) + .^7 (71/ - l)-\/M {M - !)-'] 



, r + 7l/(J/-l)-' 

 = log,/ 



71/ (71/ - ir' 



= log,/ (/• + M {M - I)-') - log,/ .1/ + log,/ (.1/ - 1) 

 which is of the form 



Cr = log,/ (/• + m) + / 

 where m and jn are factors independent of r. Mandelbrot shows 

 that the general form of the expression for CV \s> the same no matter 



