Ralston et al.: A meta-analytic approach to quantifying scientific uncertainty in stock assessments 
219 
7.00 
„ 6.00 
£ 5.00 
(D 
(0 4.00 
£ 
O 
^ 3.00 
O) 
c 
| 2.00 
a 3 
w 1.00 
0.00 
1970 1975 1980 1985 1990 1995 2000 2005 2010 
Year 
Figure 1 
Biomass time series for Pacific whiting ( Merluccius productus ) based 
on 15 historical stock assessments conducted for the Pacific Fishery 
Management Council. The bold line with square symbols represents 
the most recent stock assessment used in the meta-analysis; the 
other lines represent time series of abundance developed from ear- 
lier assessments. 
tantly, these factors contribute to varia- 
tion in all groundfish and coastal pelagic 
species stock assessments at the PFMC, 
collectively exhibit considerable variation 
among historical assessments. Moreover, 
it is unsettling to managers when stock 
size estimates fluctuate greatly from one 
assessment to the next because this fluc- 
tuation undermines confidence for scien- 
tific advice. Hence, we assert that quan- 
tifying and accounting for this source of 
uncertainty is the first and most impor- 
tant factor to consider when establish- 
ing a buffer between the OFL and the 
ABC. We recognize, however, that as the 
quantification of scientific uncertainty de- 
velops in the future it will be important 
to expand consideration to other sources 
of errors, including forecast uncertainty 
(Shertzer et ah, 2008) and uncertainty 
in estimating optimal harvest rates (e.g., 
Dorn, 2002; Prager et ah, 2003; Punt et 
ah, 2008). Hence, quantification of varia- 
tion as revealed here should be considered 
only a lower bound on total uncertainty. 
Moreover, even if both forecast and har- 
vest rate uncertainty were incorporated into our anal- 
ysis, we note that many other factors exist that would 
be difficult to quantify, including the effects of climate 
and ecosystem interactions on the estimation of OFLs. 
Quantifying biomass uncertainty 
We initially consider two types of uncertainty in bio- 
mass estimation. The first is due to estimation error, 
also termed stochastic uncertainty (Pawitan, 2001). We 
quantify this type of uncertainty using the estimated 
coefficient of variation (CV) for the terminal-year 
biomass taken from the most recent stock assessment 
conducted. In a very limited number of studies (e.g., 
Pacific ocean perch [Sebastes alutus]), full Bayesian 
integration of uncertainty with Monte Carlo Markov 
Chain analysis has been achieved. However, on the 
U.S. west coast such cases are the exception. Hence, 
we report the asymptotic standard error for the esti- 
mate of terminal biomass developed by inverting the 
model’s Hessian matrix as a first-order approximation 
of variation, i.e., the observed Fisher information sta- 
tistic (Pawitan, 2001). The accuracy of this approxima- 
tion depends on how well the log-likelihood surface 
at its maximum can be approximated by quadratic 
curvature, and on proper specification of the likelihood 
components, including appropriate error distributions 
and variance weightings. 
We view this error estimate as a measure of statisti- 
cal uncertainty within a stock assessment model that is 
conditioned on all the structural assumptions embedded 
within the model. We convert the asymptotic standard 
error to a CV by simple division using the terminal 
biomass estimate as the denominator. It is important 
to note that we limit our consideration to terminal-year 
biomass because under the reauthorized MSA, quan- 
tification of scientific uncertainty is used to prevent 
“overfishing,” which occurs when 1) the current year 
catch exceeds the OFL; or 2) an updated assessment 
retrospectively indicates that fishing mortality exceeded 
F ' MSY . Overfishing per se occurs only on an annual ba- 
sis, although the chronic effect of overfishing results 
in stocks becoming depleted, and if fishing mortality is 
substantially greater than F MSY , a stock will eventually 
become overfished. 
The second type of uncertainty can be thought of as 
among-assessment variation, which is attributable to a 
wide variety of factors, many of which represent a sig- 
nificant form of model or inductive uncertainty (Pawi- 
tan, 2001). Assertion of asymptotic or dome-shaped 
selectivity patterns is one example, as is incorpora- 
tion of age-dependent natural mortality. Assumptions 
regarding such structural issues will often change 
from one assessment to the next. Likewise, values 
for biologically important parameters (e.g., natural 
mortality or spawner-recruit productivity), which are 
prespecified when using auxiliary information (or 
expert judgment), may change, or an entire new data 
series may be incorporated into the assessment as new 
data become available. Beyond such changes in model 
specification, among-assessment variation includes 
other sources of variability due to, for example, dif- 
ferences in the reviewers who evaluated, suggested 
changes to, and ultimately approved an assessment 
model. 
To quantify among-assessment variability we as- 
sembled time series of biomass from historical as- 
sessments of groundfish and coastal pelagic species 
