MENDELSSOHN: ESTIMATING POPULATION SIZES 



The relationship between this "smoothness 

 priors" approach, "smoothing spHnes", and other 

 penalized likelihood methods is discussed further for 

 a variety of contexts in Brotherton and Gersch 

 (1981), Kitagawa and Gersch (1984, 1985, 1988), 

 Ansley and Kohn (1986), and Kohn and Ansley 

 (1987, 1988). Wahba (1977) and O'Sullivan (1986) 

 discussed the relationship between generalized 

 cross-validation, penalized likelihood functions, and 

 determining the tradeoff between smoothness and 

 fit. 



For k = 0, Equation (29) models recruitment as 

 a random variable around a fixed but unknown mean 

 value. For k = 1, Equation (29) models recruitment 

 as a random walk with unknown mean level and drift 

 (variance). Higher values of /c have similar interpre- 

 tations. Values of A' higher than two or three rarely 

 need to be considered, since these include the dis- 

 crete equivalent of cubic splines. Cubic splines can 

 approximate most functionals (in this case of time) 

 to a reasonable degree of accuracy. 



For this example, I assume k = 1, so that 



r(0 = r{t - 1) + w{t) 



(31) 



which is a random walk with unknown variance. (A 

 more complete analysis of this data would probably 

 also include an irregular stationary term as in Equa- 

 tion (30) and determine the "best" order of differ- 

 encing using a given criterion.) Equation (31) can 

 be incorporated into the state space model by let- 

 ting the (1,1) element of the matrix F be equal to 



1. The matrices H{t) are diagonal matrices whose 

 values are calculated from table 13 in Parrish and 

 MacCall (1978). Because I am assuming that the 

 estimates of F are known, then the value of Q for 

 the M step is maximized as 



T-\S,iO) - S,{1)F^ - FS,il) + FS,_,{0)F^]. 



(32) 



As in Parrish and MacCall (1978), I treated age 

 groups 4 through 6 as fully selected by the fishery, 

 and will refer to these age groups as "adults". 

 Similarly, I refer to the number of age-1 fish at the 

 start of the season as the number of recruits. I 

 assume that F and H{t) are known, so the estima- 

 tion problem is reduced to determining the means 

 of the initial population sizes and the values of the 

 two covariance matrices Q and R. 



The resulting maximum likelihood estimates of Q 

 and R (Tables 1, 2) show that the variances of the 

 error terms differ by up to two orders of magnitude, 

 hardly meeting the usual assumption of equal vari- 

 ances. Moreover, the covariances (expressed as cor- 

 relations in the tables) are quite high, so that using 



- as a weighting factor will not be adequate. The 

 a 



predicted, filtered, and smoothed estimates of the 

 adults (Fig. 1) are very similar, reflecting that the 

 errors have been "filtered out" over time by the 

 population dynamics. 



Table 1 .—Estimated values of the matrix O presented as a variance-correlation matrix. The diagonal 

 terms are the variances, and the off-diagonal terms are the cross-correlations. 



Table 2. — Estimated values of the matrix R presented as a variance-correlation matrix. 



623 



