Phase 4: Interpretation and Utility 



Guideline 13: Data Quality Objectives 



The discriminatory ability of the indicator should be evaluated against program data quality objectives and 

 constraints. It should be demonstrated hov\/ sample size, monitoring duration, and other variables affect 

 the precision and confidence levels of reported results, and how these variables may be optimized to attain 

 stated program goals. For example, a program may require that an indicator be able to detect a twenty 

 percent change in some aspect of ecological condition over a ten-year period, with ninety-five percent 

 confidence. With magnitude, duration, and confidence level constrained, sample size and extraneous 

 variability must be optimized in order to meet the program's data quality objectives. Statistical power 

 curves are recommended to explore the effects of different optimization strategies on indicator 

 performance. 



Traditional power analyses are employed in hypothesis testing to evaluate the probability of failing to reject 

 the null hypothesis when the alternative hypothesis is true. This is called a "Type II error" and has been 

 described as "not seeing enough in the data" by Anderson (1966 as cited in Keppel and Saufley 1980). 



Power is then the probability that a correct decision is made {i.e., the null hypothesis is rejected when the 

 alternative hypothesis is true). Power analyses are recommended as part of the experimental design process 

 to determine the number of samples needed in order to make a correct decision with a given level of power. 

 Afterward, power analyses may be used to determine the probability that a correct decision was made given 

 the number of samples and the sample variance. 



Power analyses were Included in the design of EMAP-E but were different from traditional analyses in that we 

 were evaluating the power to detect a trend rather than the power to reject a null hypothesis. EMAP-E set a 

 performance goal of detecting a 2% per year change in the province-wide proportion of area that exceeds a 

 pre-specified indicator threshold value over a period of 12 years with a probability exceeding 0.8 (Larsen et 

 al. 1995, Heimbuch et al. [in review]). In order to test whether EMAP-E is capable of meeting this target, 

 power analyses were performed to construct scenarios for detecting a 1% to 3% change over 12 years. 



The power to detect a 2% trend over 12 years is calculated by first estimating the variance. The overall 

 variance consists of two components: a spatial component and a component that is dependent on the number 

 of years over which you wish to estimate a trend. Spatial variance was estimated for the four years during 

 which data were collected. We computed power curves for 1 % to 3% change per year (p = 0.01 , 0.01 5, 0.02, 

 0.025, 0.03) for years 6 to 12 at a = 0.10 according to the methods presented in Heimbuch et al. (in review) 

 for EMAP-E (Figure 3-8). According to the results presented here for the Louisianian Province, EMAP-E has 

 met its performance goal of detecting a 2% change per year in the proportion of area with degraded benthic 

 communities. 



Guideline 14: Assessment Thresholds 



To facilitate interpretation of indicator results by the user community, threshold values arranges of values 

 should be proposed that delineate acceptable from unacceptable ecological condition. Justification can 

 be based on documented thresholds, regulatory criteria, historical records, experimental studies, or 

 observed responses at reference sites along a condition gradient. Thresholds may also include safety 

 margins or risk considerations. Regardless, the basis for threshold selection must be documented. 



3-22 



