Phase 4. Interpretation and Utility 



Guideline 13: Data Quality Objectives 



The discriminatory ability of tlie indicator should be evaluated against program data quality objectives 

 and constraints. It should be demonstrated how sample size, monitoring duration, and other variables 

 affect the precision and confidence levels of reported results, and how these variables may be optimized 

 to attain stated program goals. For example, a program may require that an indicator be able to detect a 

 twenty percent change in some aspect of ecological condition over a ten-year period, with ninety-five 

 percent confidence. With magnitude, duration, and confidence level constrained, sample size and 

 extraneous variability must be optimized in order to meet the program's data quality objectives. Statistical 

 power curves are recommended to explore the effects of different optimization strategies on indicator 

 performance. 



Performance Objectives 



1 . Demonstrate the capability of the indicator to distinguish classes of ecological condition within the 

 proposed monitoring framework. 



2. Demonstrate the capability of the indicator to detect trend in condition change within the proposed 

 monitoring framework. 



The capacity to estimate status and detect trend in condition is primarily a function of variability. Variability is 

 due in part to natural differences that occur across a set of sampling sites (Guidelines 8 through 1 1 ), and also 

 to differences in the intensity of human disturbance across those sites (Guideline 1 2). An indicator can have 

 low variability (and thus high statistical power), but poor discriminatory capability because it cannot discern 

 differences in intensities of human disturbance. However, high variability serves to reduce the discriminatory 

 capability of an indicator. 



Specific performance criteria for the indicator to detect trend in ecological condition have been developed for 

 the proposed monitoring framework (Table 4-18). These criteria were examined using several power curves 

 for the indicator to evaluate the effects of coherent variation across years, magnitude of trend, and sample 

 size (Fig. 4-9). These curves were developed using the initial variance component estimates from the 1993- 

 1 994 MAHA study (Guidelines 1 and 1 1 ) and the approach described by Larsen ef a/. (1 995) and Urquhardt 

 et al. (1998). Derived estimates of the coherent variation across years were not used because they are 

 based on only two years of data. Instead, to provide a range of possible scenarios, values of coherent 

 variation (S^ ^J were substituted to range from 0-100, where 100 is approximately 1.7 times the within-year 

 variance (S^^^^.^^^,). Four different magnitudes of trend were also evaluated, ranging from 0.5 to 2 indicator 

 points per year (equal to 0.5 - 4% per year for an indicator score of 50 points). This represents a potential 

 trend in the indicator score of 5 to 20 points over a 10-year period. 



With respect to estimating status, the indicator satisfies the performance criterion (Table 4-18) under the 

 conditions specified in Figure 4-9 (A). After 4 years of monitoring, the standard error of the indicator score 

 ranges between 1 and 2 points (depending on sample size), which would provide 95% confidence intervals 

 of about ±2 to ±4 points (which is less than 1 0% of the proposed impairment threshold of 50 points [Table 4- 

 1 8]). Intervals computed for a = 0. 1 (90% confidence intervals) would be smaller. With continued monitoring, 

 the standard error of the estimate is stable through time. 



4-35 



