34 Henry Quastler 



message can be used for error checking the more effectively the more evenly it 

 is related to all parts of the message. 



It is always possible to achieve reliability, in the presence of noise, by the 

 use of redundant information; in fact, one can approach perfect reliability 

 arbitrarily closely if one is willing to provide enough redundant information. 

 The amount of redundant information needed, for a given noise level and a 

 given desired reliability, will depend on the efficiency of coding. The ideal 

 relation between noise level and redundant information needed is formulated 

 in Shannon's fundamental Theorem of the Noisy Channel. This theorem can be 

 stated as follows: if a certain amount of information is to be transmitted with 

 perfect reliability in the presence of noise, then it is necessary to provide at 

 least as much redundant information as the amount of equivocation introduced 

 by the noise ; furthermore, this amount will be sufficient if the coding is maximally 

 efficient. 



There exist several proofs of this theorem; none of them is easy to follow, 

 and all are existence proofs — that is, they prove that an error-checking code 

 exists which will fulfill the requirements, but they do not say how to construct 

 it. In fact, perfectly efficient error-checking codes seem to be realizable only in 

 a few special cases; however, close approximations to ideal efficiency are easily 

 obtained if it is permissible to use message blocks of great length (12). 



The economics of error-checking are dominated by three factors: 



(I) the frequency and costliness of errors 



(II) the cost of adding redundant information 



(III) the availability and costliness of checking procedure (encoding and 

 decoding). 



The work of Shannon and his followers has dealt with one particular situation : 

 encoding and decoding procedures are supposed to be reliable and gratis, the 

 error frequency is to be reduced to almost zero, and redundant information is 

 supposed to be used as sparingly as possible. As long as the theory is not 

 completed even for this case, one cannot expect to develop a more general theory. 

 Some qualitative notions of what it will entail can be gathered from a considera- 

 tion of a much-used, and presumably well developed communication system, 

 namely, printed language. Symbols are gathered into various checking units 

 (words, sentences, paragraphs, chapters) ; on each level, there operate constraints 

 which will help to locate and correct errors. For instance, this sentence will be 

 read corretly even though one letter has been onitted and one word misspelled. 

 It 3eems that the redundancy per letter, in a coherent English text, is about 60 

 per cent. Paragraphs are constructed in such a way that the sense can be 

 grasped even if whole words or even sentences are missing or perturbed, and 

 the essence of a whole chapter is, in general, understandable even if a whole 

 paragraph should be left out. 



Actual Communications System 



So far we have dealt with two-part systems in a purely abstract way. 'Sources' 

 and 'destinations' are defined simply by the states which they can assume. 

 'Channels' are tables of conditional probabilities; in the simplest case, the 

 channel is a kind of telephone book which associates every input to some 



