510 THE BELL SYSTEM TECHNICAL JOURNAL, MAY 1952 



their invulnerability to noise. For example, it is clear that the alphabet 

 3, 15 is better than 2, 8. However, without further information about 

 the channel, such as knowledge of p, there is no reasonable way of choos- 

 ing between 3, 15 and 3, 7. 



3. Large Alphabets 



We have been unable to prove that there are error correcting alphabets 

 which signal at rates arbitrarily close to C while maintaining an arbi- 

 trarily small probability of error for any letter. A result in this direction 

 is the following theorem. 



Theorem 2. Lei any positive e and 8 he given. Given a channel with p < \ 

 there exists an error correcting alphabet which can signal over the channel 

 at a rate exceeding Ro — e where 



R,= l + 2p log 2p-\- (1 - 2p) log (1 - 2p) 



bits per digit and for which the probability of error in any letter is less 

 than 8. 



Proof 



The probability of error in any letter is the sum on the left of (4) . This 

 is a sum of terms from a binomial distribution which, as is well known, 

 tends to a Gaussian distribution with mean Dp and variance Dp(l — 

 p) for large D. Hence there is a constant ^(5) such that all k error cor- 

 recting alphabets with sufficiently large D have a letter error proba- 

 bility less than 8 provided 



k>Dp + A(8) (Dp(l - p)y" (6) 



Let k{D) be the smallest integer which satisfies (6) and consider an 

 alphabet which corrects k(D) errors and contains Ko(D, k(D)) letters. 

 By Equation (5) and the lower bound of Theorem 1, this alphabet signals 

 at a rate R(D) satisfying 



1 - ^ log N{D, 2k{D)) < R{D). 



Since p < i, 2k(D) < D/2 for large D and hence 



N(D, 2k{D)) < C2k{D) + l)Co.-2HD,. 



Then an application of Stirling's approximation for factorials shows that 

 as Z) -^ c» 



1 -^logiV(A2fc(£>))-^/?o. 



