215 



Responses to Follow-up Questions 

 House Agriculture Committee Hearing (6/17/93) on 

 Priority-Setting and Agricultural Research Facilities 



Daryl E. Chubin 



MichaelJ. Phillips 



Office of Technology Assessment 



July 6, 1993 



1. How do you suggest we can better evaluate research quality? What role should Congress 



play in this process? 



Aldiough books have been written on the subject of "quality," we have gotten little beyond the 

 maxim "1 know it vAtsn I see it." As OTA points out in Federally Funded Research: Decisions for a 

 Decade, the elusive nature of research makes measuring its outcomes very difficult (pp. 242-247). OTA 

 also notes that Congress ideally needs information to guide its investments, i.e., to decide in advance about 

 research quality and other outcomes. 



Congress could follow either or both of the following approaches to improve matters: fund 

 research on "evaluation" and "assessment" of research programs and projects; and/or ask the agencies to 

 develop criteria to measure outcomes of the research they support. The National Science Foundation 

 modestly supports such investigations, while the National Institutes of Health reserve up to 1 percent of 

 each institute's annual budget for evaluation activities. The General Accounting Office (GAO/PEMD-93- 

 13) just reported, however, that the evaluation set-aside at the Public Health Service has not worked to 

 inform the Congress as intended. 



Any grants-making agency that employs peer review in the award of research monies might be 

 expected to have, or to develop, a system for measuring the outcomes of that support. While it remains 

 hard to determine the returns on investment in research, some accountability is necessary. Congress may 

 need to remind the research agencies that it is important to find creative wa>'s of demonstrating the quality 

 of the work they support. 



2. Given recent budgetary pressures, how do we determine which universities should be our 



research intensive universities? Cleariy they cannot all be. 



Depending on how one counts, there are 150-200 research-intensive universities in the U.S. 

 However, the support is even more concentrated, with 20 universities accounting for almost 40 percent of 

 Federal R&D expenditures, 60 accounting for 70 percent, and 1 00 accounting for 85 percent. To reiterate 

 from OTA's A New Technological Era for Agricultural Research (p. 4 1 2), experiment stations in 12 

 States account for nearly half the total research funds available to experiment stations, more than two- 

 thirds of USDA's competitive grants, over 60 percent of all competitively-awarded funds fixxn Federal 

 agencies other than USDA, and nearly 60 percent of all funding from industry. In short, we know where 

 the researd) action is. Very little change in rank occurs among the top 50; the scramble to become major 

 players is amof^ the next 50 institutions. 



