VERIFICATION OF WEATHER FORECASTS 
By GLENN W. BRIER and ROGER A. ALLEN 
U.S. Weather Bureau, Washington, D. C. 
INTRODUCTION 
Verification of weather forecasts has been a con- 
troversial subject for more than sixty years and has 
affected nearly the entire field of meteorology. This 
paper will discuss some of the important reasons for 
this controversy and attempt to show that much of 
the existing confusion disappears when a careful analy- 
sis is made of the objectives of forecasting and verifica- 
tion. A number of verification systems that have been 
used will be described, but it is beyond the scope of this 
paper to make a complete survey of verification prac- 
tices or of the literature on the subject. For the latter 
purpose the reader is referred to articles by Bleeker 
[1], Muller [6], and the U. S. Weather Bureau [10], 
which contain extensive bibliographies. 
Definition of the Problem. Verification is usually 
understood to mean the entire process of comparing 
the predicted weather with the actual weather, utiliz- 
ing the data so obtained to produce one or more indices 
or scores and then interpreting these scores by com- 
paring them with some standard depending upon the 
purpose to be served by the verification. In the dis- 
cussion which follows, it will be assumed that both 
forecasts and observations have been expressed ob- 
jectively so that no element of judgment enters mto 
the comparison of forecast with observation, and it 
will be assumed further that errors of observation are 
unimportant. These assumptions are discussed in a 
subsequent section. The selection and interpretation 
of an index or score which will meet the objectives 
of the verification study usually constitute the most 
difficult part of the problem since, as will be shown, 
practical considerations often require that the score 
fulfill a number of requirements and furnish informa- 
tion on a number of different characteristics of the 
forecasts. Selection of arbitrary scores intended to meas- 
ure a number of parameters usually leads to difficulty 
in the last step, interpreting the score. 
PURPOSES OF VERIFICATION 
One of the earliest purposes of forecast verification 
was to justify the existence of the newly organized 
national weather services, and thus the question of 
verification immediately became a football to be kicked 
around by the supporters and opponents of the na- 
tional weather services. Some, such as Klein [5], claimed 
that the official synoptic forecasts had little or no 
practical value and produced verification figures im- 
tended to prove their point. Others, for instance 
Schmauss [7], claimed that the weather forecasts could 
not be subjected to rigorous statistical tests or that 
the tests that had been performed had no meaning. 
Today the value of the national weather services is so 
widely accepted that this particular purpose of verifica- 
tion no longer is of much importance. However, the 
effects of this controversy still show their influence by 
often preventing a realistic attempt to solve some of 
the problems of forecast verification since some mete- 
orologists object a priori to any scoring system that 
does not produce high enough figures to make the 
forecasts appear favorable in the eyes of the public or 
government appropriating bodies. 
Economic Purposes of Verification. Since the sole 
purpose of practical forecasting is economy of life, 
labor, and dollars, it would seem that one of the chief 
purposes of verification is to determine the economic 
value of the forecasts which have been made. But such 
an evaluation, especially for the whole economy, is 
difficult or impossible since the uses and users of fore- 
casts are so diverse and ramified. Forecasts that may 
have considerable value for one user may have little 
or no utility for another user or may actually have a 
negative value if their accuracy is low. The measure- 
ment of the economic value of the forecast is thus a 
separate project for each user, and this is usually 
impracticable or impossible because the individual has 
neither the essential economic data nor the facilities 
to make such an evaluation. Thus a farmer may feel 
that an accurate weather forecast enabled him to make 
some saving in seed cost and planting labor, but he 
may be unable to estimate the cost of his own labor, and 
the total saving might depend upon the value of the 
harvested crop which is, of course, unknown until 
harvest time. Sometimes such economic evaluation of 
the forecasts is possible, the necessary data being found 
in the cost accounting and financial structure of the 
particular business [2]. However, since evaluation in 
economic terms is usually impossible, it is most often 
desirable to determine the reliability of weather fore- 
casts by measuring their approach to the truth and 
expressing the result in terms of some arbitrary scale 
such as errors in degrees Fahrenheit or per cent of hits. 
The user of the forecast can then interpret the informa- 
tion in terms of his operations. Thus an electric power 
company may want to know when thunderstorms are 
forecast for a given area and what the likelihood is that 
the forecast will be correct, or, on the other hand, what 
per cent of observed thunderstorms in an area will be 
correctly forecast. 
Administrative Purposes of Verification. One of the 
most useful purposes of verification is to determine 
the relative ability of different forecasters. In a weather 
organization the relative ranks of forecasters may be 
desired for a number of reasons, such as selecting 
promising personnel for further training in forecasting, 
selecting able men for difficult or specialized forecasting 
841 
