217 
A mefa-analytic approach to quantifying 
scientific uncertainty in stock assessments 
Email address for contact author: Steve.Ralston@noaa.gov 
National Marine Fisheries Service 
Southwest Fisheries Science Center 
Fisheries Ecology Division 
1 10 Shaffer Road 5 
Santa Cruz, California 95060 
University of Washington 
School of Aquatic and Fishery Science 
1122 NE Boat Street 
Seattle, Washington 98195 
National Marine Fisheries Service 
Fishery Resource Analysis and Monitoring Division 
2725 Montlake Blvd. East 
Seattle, Washington 98112 
Pacific Fishery Management Council 
7700 NE Ambassador Place, Suite 101 
Portland, Oregon 97220 
National Marine Fisheries Service 
Southwest Fisheries Science Center 
Fisheries Resource Division 
8604 La Jolla Shores Drive 
La Jolla, California 92037 
Abstract — Quantifying scientific 
uncertainty when setting total allow- 
able catch limits for fish stocks is a 
major challenge, but it is a require- 
ment in the United States since 
changes to national fisheries legis- 
lation. Multiple sources of error are 
readily identifiable, including estima- 
tion error, model specification error, 
forecast error, and errors associated 
with the definition and estimation 
of reference points. Our focus here, 
however, is to quantify the influ- 
ence of estimation error and model 
specification error on assessment 
outcomes. These are fundamental 
sources of uncertainty in developing 
scientific advice concerning appro- 
priate catch levels and although a 
study of these two factors may not 
be inclusive, it is feasible with avail- 
able information. For data-rich stock 
assessments conducted on the U.S. 
west coast we report approximate 
coefficients of variation in terminal 
biomass estimates from assessments 
based on inversion of the assessment 
of the model’s Hessian matrix (i.e., 
the asymptotic standard error). To 
summarize variation “among” stock 
assessments, as a proxy for model 
specification error, we characterize 
variation among multiple histori- 
cal assessments of the same stock. 
Results indicate that for 17 ground- 
fish and coastal pelagic species, the 
mean coefficient of variation of termi- 
nal biomass is 18%. In contrast, the 
coefficient of variation ascribable to 
model specification error (i.e., pooled 
among-assessment variation) is 37%. 
We show that if a precautionary prob- 
ability of overfishing equal to 0.40 is 
adopted by managers, and only model 
specification error is considered, a 
9% reduction in the overfishing catch 
level is indicated. 
Manuscript submitted 20 September 2010. 
Manuscript accepted 23 February 2011. 
Fish. Bull. 109:217-231 (2011). 
The views and opinions expressed 
or implied in this article are those of the 
author (or authors) and do not necessarily 
reflect the position of the National Marine 
Fisheries Service, NOAA. 
Stephen Ralston (contact author)' 
Andre E. Punt 2 
Owen S. Hamel 3 
John D. IDeVore 4 
Ramon J. Corner 5 
It has long been recognized that 
precautionary measures in fisheries 
management should be related to the 
amount of uncertainty in the science 
that is used to evaluate stock status 
(Caddy and McGarvey, 1996; FAO, 
1996). However, few fisheries jurisdic- 
tions have adopted precautionary har- 
vest control rules that are designed to 
reduce “risk-neutral” point estimates 
of catch based on the amount of uncer- 
tainty in the estimates, although at 
least two examples of this type of 
precautionary approach exist in the 
management of marine mammal pop- 
ulations. The International Whaling 
Commission has adopted a manage- 
ment procedure for baleen whales 
where, for example, a posterior distri- 
bution for the output of a harvest con- 
trol rule is computed, and the catch 
limit is set close to the 40 th percen- 
tile of the distribution (IWC, 1999; 
Punt and Donovan, 2007). Likewise, 
with the potential biological remov- 
als method (Wade, 1998), the level of 
marine mammal take at which man- 
agement action must occur is based on 
the 20 th percentile of the most recent 
estimate of abundance. 
The reauthorization of the Mag- 
nuson-Stevens Fishery Conserva- 
tion and Management Act (MSA) in 
2006 changed the requirements for 
how management actions are devel- 
oped for U.S. fisheries. The eight Re- 
gional Fishery Management Coun- 
cils are now required to set annual 
catch limits (ACLs) for all managed 
stocks that are “in the fishery.” Na- 
tional Standard Guidelines have now 
been developed to assist in the im- 
plementation of the reauthorized act 
(Federal Register, 2009), which de- 
fines two sources of uncertainty that 
must be considered when establish- 
ing ACLs: 1) scientific uncertainty, 
including error pertaining to both 
the data and to parameter estima- 
tion; and 2) management uncertainty, 
which represents uncertainty in the 
efficacy of management practices that 
are designed to ensure that harvest 
limits are not exceeded. The focus of 
this study is on the first of these two 
sources of uncertainty. 
Defining “scientific uncertainty” 
is not trivial. It is therefore not sur- 
prising that a variety of approaches 
have been taken to quantifying un- 
