VERIFICATION OF WEATHER FORECASTS 
hedge by playing the middle of the range. Thus a fore- 
caster may feel that the minimum temperature during 
the night will be near 40F if the cloud cover remains 
but will be near 30F if the skies clear. If he is being 
verified on the basis of the RMSE, he might tend to 
choose 35F in cases of doubt even though he may be 
quite sure that the temperature will not be close to 35F. 
These scores can also be used to verify prognostic 
pressure-patterns by comparing the forecast and ob- 
served pressures or pressure gradients at a number of 
sample stations or points on the map. 
Correlations. The correlation coefficient is a sta- 
tistie which measures association between two sets of 
values, such as forecast and observed temperatures. 
One simple formula for the coefficient is 
dul NFO: — (2 F) (OO) 
VN Fi - OOF)? VN DO? - (00)? 
When used as a verification score it has the advantage 
of being relatively free from the fault of influencing the 
forecaster in an undesirable way. However, it is insensi- 
tive to any bias or error in scale that a forecaster may 
have. Thus the centigrade and Fahrenheit scales are 
perfectly correlated, but a person would not use one 
scale to verify forecasts made using the other scale. 
The correlation coefficient is also difficult to interpret 
for the purpose of making operating decisions and is 
subject to abuse of interpretation, such as attaching too 
much significance to the value of a coefficient obtained 
from a small sample. It is influenced by trends that 
exist in both series so it is usually desirable to compute 
it by using departures from normal. 
Verification of Probability Statements. There appears 
to be one situation where it is possible to devise a veri- 
fication scheme that cannot influence the forecaster in 
any undesirable way. This is the case when the fore- 
casts are expressed in terms of probability statements. 
Suppose that on each of N occasions an event can occur 
in only one of 7 possible classes and on one such occa- 
sion 2, fi, fix,.-..-,fir represent the forecast proba- 
bilities that the event will occur in classes 1, 2,..., 7, 
respectively. If the r classes are chosen to be mutually 
exclusive and exhaustive, >> fi; = 1 for each and every 
j=1 
i = Wy oo wien Wie 
A score P can be defined by 
rN 
[Ps x 2 z (ig = E;;)’, 
7=1 i=1 
where #;;; takes the value 1 or 0 according to whether 
the event occurred in class j or not. For perfect fore- 
casting this score will have a value of zero and for the 
worst possible forecasting a value of 2. Perfect fore- 
casting is defined as “correctly forecasting the event to 
oceur with a probability of unity or 100 per cent confi- 
dence.” The worst possible forecast is defined as 
“stating a probability of unity or certainty for an event 
that did not materialize” (and also, of course, “‘stating 
a probability of zero for the event that did materialize”). 
It can be shown that if pi, po, ps, ..-, pr are the 
845 
respective climatological probabilities of classes 1, 2, 
3,...,7 then in the absence of any forecasting skill 
the best values to choose for f;; will be p; for all N 
occasions. This will minimize the score P for constant 
values of fi; = fo; = ... fn; and the expected value of 
the score will be 
E(P) = 1 — 2p}. 
A 
To illustrate the procedure consider the following 
table of ten actual forecasts of “rain” or “no rain” in 
which a probability or confidence statement was made 
for each forecast. 
Taste IV. ExampeLe or Forecasts STATED IN TERMS OF 
PROBABILITY 
Rain No rain 
Occasion _ 
pebabiily | Observed | SESSEEH, | Observed 
1 0.7 0 0.3 1 
2 0.9 1 0.1 0 
3 0.8 1 0.2 (0) 
4 0.4 1 0.6 0 
5 0.2 0 0.8 1 
6 0.0 0 1.0 1 
7 0.0 0 1.0 1 
8 0.0 0 1.0 1 
10 0.1 0 0.9 1 
In the table, unity is placed in the “rain” column if rain 
occurs. If the event is “no rain,” unity is placed in the 
‘no rain” column. The score P is therefore 
P = 2y{0.72 + 0.72 + 0.12 + 0.12 + 0.22? + 0.22+... 
0.12 + 0.123} = 0.19. 
Since rain occurred 340 of the time, the minimum (or 
best) score that could have been obtained by making the 
same forecast every day would be 
Prin = 1 (0:3? == 0:77) = 042: 
Thus the forecaster is encouraged to minimize his 
score by getting the forecasts exactly right and stating 
a probability of unity. If he cannot forecast perfectly, 
he is encouraged to state unbiased estimates of the 
probability of each possible event. On the other hand, 
TaBLE V. VERIFICATION OF A SERIES OF 85 Forecasts Ex- 
PRESSED IN TERMS OF THE PROBABILITY OF RAIN 
Forecast probability of rain Observed proportion of rain cases 
0.00-0.19 0.07 
0.20-0.39 0.10 
0.40-0.59 0.29 
0.60-0.79 0.40 
0.80-1.00 0.50 
with complete absence of knowledge or forecasting 
skill he is encouraged to predict the climatological 
probabilities and not just forecast the most frequent 
class. 
