656 BELL SYSTEM TECHNICAL JOURNAL 



and consequently the sum is increased. Thus the various possible subdivi- 

 sions form a directed set, with R mono tonic increasing with refinement of 

 the subdivision. We may define R unambiguously as the least upper bound 

 for the R\ and write it 



i^ = i // P(.,y) log ^>g) </.-/.. 



This integral, understood in the above sense, includes both the continuous 

 and discrete cases and of course many others which cannot be represented 

 in either form. It is trivial in this formulation that if x and u are in one-to- 

 one correspondence, the rate from w to y is equal to that from x to y. If v 

 is any function of y (not necessarily with an inverse) then the rate from x to 

 y is greater than or equal to that from x to i) since, in the calculation of the 

 approximations, the subdivisions of y are essentially a finer subdivision of 

 those for v. More generally if y and v are related not functionally but 

 statistically, i.e., we have a probability measure space (y, v), then R{x, v) < 

 R{x, y). This means that any operation applied to the received signal, even 

 though it involves statistical elements, does not increase R. 



Another notion which should be defined precisely in an abstract formu- 

 lation of the theory is that of ''dimension rate," that is the average number 

 of dimensions required per second to specify a member of an ensemble. In 

 the band limited case 2W numbers per second are sufficient. A general 

 definition can be framed as follows. Let/a(0 be an ensemble of functions 

 and let prlfaiO , fM\ be a metric measuring the "distance" from/a to /^ 

 over the time T (for example the R.M.S. discrepancy over this interval.) 

 Let N{e, 8, T) be the least number of elements / which can be chosen such 

 that all elements of the ensemble apart from a set of measure 5 are within 

 the distance e of at least one of those chosen. Thus we are covering the 

 space to within e apart from a set of small measure 5. We define the di- 

 mension rate X for the ensemble by the triple limit 



. ... ^. ^. log 7\^(e, 5, r) 

 X = Lim Lim Lim -^^ ^— ^ — . 

 5_»o €-»o r-»oo i log e 



This is a generalization of the measure type definitions of dimension in 

 topology, and agrees with the intuitive dimension rate for simple ensembles 

 where the desired result is obvious. 



