51 



26 / IMPROVISG THE ALLOCATION PROCESS 



the supported institutions choose the researchers and their projects. Evaluations of 

 that system of formula-grant allocation have not given high marks to its responsi\e- 

 ness or the qualit)' of the resulting research.^' Other federal funding is awarded 

 competitively to research centers, which in turn distribute the funding among 

 individual researchers and groups. 



There is benefit to having a varierv' of approaches to supporting FS&T, espe- 

 cially because mission agencies have specialized assignments to fulfill. However, the 

 committee believes that fiscal constraint makes it important to level the playing 

 field. Competitive merit review should therefore be increased relative to other 

 mechanisms for awarding FS&T funds. Merit review is best exemplified by the 

 processes used at the NSF and NIH, that is, the use of external peer review to iden- 

 tif\' and select the best proposals for individual research projects as part of a review 

 process based on competition and expert evaluation of merit criteria. That ap- 

 proach enables those two agencies to choose the best performers. Accordingly, use 

 of competitive merit review to allocate federal funding should be the default pre- 

 sumption, supplemented with other mechanisms for inherently governmental 

 functions that cannot be accomplished through competitive merit review. 



RECOMMENDATION 11. Evaluations of research and develop- 

 ment programs and of those performing and sponsoring the 

 work also should incorporate the views of outside evaluators. 



Technical merit, which is the primar)' criterion used in performance reviews 

 of research agencies and programs as well as proposals, is best evaluated by inde- 

 pendent scientific or engineering peers. Agency performance review systems differ 

 in the extent to which they use external reviewers, but there are two compelling 

 rea,sons to reh' heavil\' (althougli not exclusively) on external reviews. First, because the 

 federal government funds most research and development outside its own laborato- 

 ries in industr)', universities, and other nongovernment research institutions, most of 

 the qualified reviewers are outside government. Second, external reviewers are a 

 more di\'ersified source of opinion and can bring a wider range of experiences to the 

 review process compared with federal agency personnel. Where needs are highh^ 

 specific, such as development of a stealth aircraft or rapid response to an emerging 

 infection, external reviews are still useful, although they may have to be retrospec- 

 tive rather than prospective. Government officials must make the final decision. 



Recent changes across the federal government emphasize improving perfor- 

 mance review and program evaluation. Indeed, according to the Government 

 Performance and Results Act (GPRA) of 1993 (Public Law 103-62), even,' federal 

 agenc} must have performance goals and measures for its programs (including FS&T 

 programs) by 1997 for its Fiscal Year 1999 budget submission. It will be difficult to 

 apply GPRA requirements to research and development activities because, by their 

 nature, they are long-term and their impacts are diffuse and hard to measure. ^■^ 



Any system to allocate resources should be guided by explicit goals, express- 

 ing the underlying philosophy and criteria for evaluating performance. But a clear 



