Predictability and Accuracy for Decision Makers

Multiple linear regression techniques with ENSO variables were done on Florida dry season storminess (at the 1012 and 1010mb thresholds), dry season jet stream anomaly, and mean dry season MSLP. All showed skill, but the 1012 mb storminess was found to be the best overall measure of storminess over the dry season months.To assist potential users in evaluating the experimental dry season forecasts, charts of seasonal storminess predictions plotted against actual storminess numbers are presented on the left hand side of the folowing table. These charts are useful for getting a feel for season- to-season variability, and a general sense of the accuracy of seasonal predictions, but they are difficult to use quantitatively. A better method of looking at the storminess predictions is to use Taylor-Russell Diagrams (on the right hand side of the table) as explained in:

Pielke, R. A. Jr., et al, 2000: Prediction - Science, Decision Making, and the Future of Nature.
     Island Press. 405 pages. ISBN 1-55963-776-5.

Taylor-Russell Diagrams: The scatterplot charts in the right-hand column of the following table show predicted dry season storminess (x-axis) versus observed dry season storminess (y-axis) for the Florida region from the 1960-2000 and 1980-2000 equations. These charts are designed to depict the uncertainty in the experimental predictions and assist in the decision making process. The horizontal line across the chart represents a boundary between situations requiring action, which are above the line, and situations requiring no action, which are below the line. This line is called the criterion. If the actual event exceeds the criterion, then some sort of preventative or protection action is required. The vertical line on the chart is the decision cutoff. Because decision makers cannot wait for an event to occur to take action, decisions must be made based on a prediction. It has been recognized that this type of chart can apply to almost any policy problem where decisions must be made in the face of uncertainty.

The two lines on the Taylor-Russell diagrams divide the scatterplot into four regions. The regions are labeled according to standard decision research terminology:

-True Positive: Appropriate action is taken. For example, people are warned of severe weather, and it actually occurs.

- False Positive: Inappropriate action is taken. This is often called false alarm. For example, people are warned of severe weather, but none occurs.

- False Negative: Action should have been taken but wasn't. For example, people were not warned of severe weather, but severe weather occurs.

-True Negative: No action was taken, and that was appropriate. For example, no warning was issued, and the storm did not become severe.

On these Taylor-Russell diagrams the criterion and decision cutoffs have been set at normal seasonal storminess and a line representing the theoretical perfect forecast equation has been added. Thus these diagrams reflect the uncertainty of above or below normal dry season storminess from the experimental forecasts. Individual users could set these thresholds to whatever criteria was important to their particular issue.