Numerous sources have developed criteria to evaluate environmental indicators. This 

 document assembles those factors most relevant to ORD-affiliated ecological monitoring 

 and assessment programs into 15 guidelines and, using three ecological indicators as 

 examples, illustrates the types of information that should be considered under each 

 guideline. This format is intended to facilitate consistent and technically-defensible 

 indicator research and review. Consistency is critical to developing a dynamic and iterative 

 base of knowledge on the strengths and weaknesses of individual indicators; it allows 

 comparisons among indicators and documents progress in indicator development. 



Building on Previous Efforts 



The Evaluation Guidelines document is not the first effort of its kind, nor are indicator needs 

 and evaluation processes unique to EPA. As long as managers have accepted 

 responsibility for environmental programs, they have required measures of performance 

 (Reams et al. 1 992). In an international effort to promote consistency in the collection and 

 interpretation of environmental information, the Organization for Economic Cooperation 

 and Development (OECD) developed a conceptual framework, known as the Pressure- 

 State-Response (PSR) framework, for categorizing environmental indicators (OECD 

 1993). The PSR framework encompasses indicators of human activities (pressure), 

 environmental condition (state), and resulting societal actions (response). 



The PSR framework is used in OECD member countries including the Netherlands 

 (Adriaanse 1 993) and the U.S., such as in the Department of Commerce's National Oceanic 

 and Atmospheric Administration (NOAA 1 990) and the Department of Interior's Task Force 

 on Resources and Environmental Indicators. Within EPA, the Office of Water adopted the 

 PSR framework to select indicators for measuring progress towards clean water and safe 

 drinking water (EPA 1 996a). EPA's Office of Policy, Planning and Evaluation (OPPE) used 

 the PSR framework to support the State Environmental Goals and Indicators Project of the 

 Data Quality Action Team (EPA 1996b), and as a foundation for expanding the 

 Environmental Indicators Team of the Environmental Statistics and Information Division. 

 The Interagency Task Force on Monitoring Water Quality (ITFM 1995) refers to the PSR 

 framework, as does the International Joint Commission in the Great Lakes Water Quality 

 Agreement (I JC 1996). 



OPPE expanded the PSR framework to include indicators of the interactions among 

 pressures, states and responses (EPA 1995). These types of measures add an "effects" 

 category to the PSR framework (now PSR/E). OPPE incorporated EMAP's indicator 

 evaluation criteria (Barber 1 994) into the PSR/E framework's discussion of those indicators 

 that reflect the combined impacts of multiple stressors on ecological condition. 



Measuring management success is now required by the U.S. Government Performance 

 and Results Act (GPFiA) of 1993, whereby agencies must develop program performance 

 reports based on indicators and goals. In cooperation with EPA, the Florida Center for 

 Public Management used the GPRA and the PSR framework to develop indicator 

 evaluation criteria for EPA Regions and states. The Florida Center defined a hierarchy of 

 six indicator types, ranging from measures of administrative actions such as the number of 

 permits issued, to measures of ecological or human health, such as density of sensitive 

 species. These criteria have been adopoted by EPA Region IV (EPA 1996c), and by state 

 and local management groups. Generally, the focus for guiding environmental policy and 



viii 



