Aviation Verify Program: Output
The output is divided into (1) basic statistics, (2) verification statistics, and (3) advanced verification statistics. The basic statistics combine the data from all TAF beginning times (0000, 0600, 1200 and 1800 UTC), and forecast projections are only parsed into two groups, 0 to 12 hours and 12 to 24 hours. The more specific verification statistics also combine the data from all TAF beginning times but parse the forecast projections periods into the following: 0 to 3 (3), 3 to 6 (6), 6 to 12 (12), 12 to 18 (18), and 18 to 24 (24) hours (the numbers in parentheses show how the projection period is abbreviated in the output). The advanced verification statistics contain the highest level of detail, and the data are parsed by forecast projection and TAF beginning time. CSIs, which do not fit on the other pages, are listed on the advanced pages. TEMPO and PROB group evaluation is more complete in the advanced verification.
General Ceiling, Visibility, and Flight Category definitions/statistics
This section is divided into three sections for (1) ceiling (CIG), (2) visibility (VIS), and (3) flight category. Ceiling (CIG) Ceilings are verified by category. A forecast hit occurs if the TAF (MOS) ceiling category equals the observation ceiling category. These categories, which are similar to NGM MOS categories and NWS TAF amendment criteria, should not be confused with the FAA flight categories. < 200 feet 200 to 400 feet 500 to 900 feet 1000 to 1900 feet 2000 to 3000 feet 3100 to 6500 feet 6600 to 12000 feet > 12000 feet or no ceiling Note that categories 4 and 5 combine to form a single NGM MOS category (1000-3000 feet). All MOS forecasts within this category are assigned to category 4 of the above table (1000-1900 feet). Visibility (VIS) Visibilities are verified by category. A forecast hit occurs if the TAF (MOS) visibility category equals the surface observation visibility category. These categories, which are similar to NGM MOS categories and NWS amendment criteria, should not be confused with the FAA flight categories. < 0.5 statute mile 0.5 to <1 statute mile 1 to <2 statute miles 2 to <3 statute miles 3 to 5 statute miles > 5 statute miles Note that categories 3 and 4 combine to form a single NGM MOS category (1 to <3 miles). All MOS forecasts within this category are assigned to category 3 of the above table (1 to <2 miles). Flight Category  There are four Federal Aviation Administration flight categories which are defined in the following table. If the ceiling and visibility values disagree on flight category, the flight category is defined as the worst of the two. LIFR - Low Instrument Flight Rules; Visibility <1 statute mile; Ceiling < 500 feet IFR - Instrument Flight Rules; Visibility 1 to <3 statute miles; Ceiling 500 to 900 feet MVFR - Marginal Visual Flight Rules; Visibility 3 to 5 statute miles; Ceiling 1000 to 3000 feet VFR - Visual Flight Rules; Visibility > 5 statute miles; Ceiling > 3000 feet Flight category statistics appear similar to the ceiling and visibility statistics (above), but the numbers are not comparable because of differences in the number and size of the categories. Due to space limitations, Very Low Instrument Flight Rule (VLIFR) statistics are sometimes presented with the flight category statistics; however, VLIFR is not a flight category. Very Low Instrument Flight Rules (VLIFR)  VLIFR conditions are not an FAA flight category but are often the criteria for closing major airports. VLIFR conditions exist whenever the ceiling is below 200 feet or the visibility is below ½ statute mile. These lowest category statistics are also computed separately for ceiling and visibility.
Weather Statistics
The following weather types/obstructions to vision are verified: Thunderstorm (TS), Rain (RA or SHRA), Snow (SN or SHSN), Freezing precipitation (FZRA or FZDZ), and fog (BR, FG and HZ). A forecast hit occurs if the TAF and surface observation contain the same weather. Intensity forecasts are not verified, e.g., heavy rain verifies a light rain forecast. Precipitation beginning and ending remarks in the MTR are used to determine precipitation times to the nearest 5-minute verification bin.
Wind Statistics
Verification is done for four elements: Wind speed (SPD), Wind direction (DIR), Light winds (LGT), and wind gusts (GST). There is no MOS guidance for wind gusts, hence MOS comparisons are not made. Wind Direction Wind direction is not evaluated if the observed wind speed is 5 knots or less. A forecast hit is counted if the TAF wind direction is within 30 degrees of the observed direction. Wind Speed Light winds (LGT) are defined as 5 knots or less for TAFs and MOS and 6 knots or less for MTRs. A forecast 'hit' occurs when both conditions are true. For all other winds, a forecast hit occurs if the absolute difference in TAF and MTR wind speed is less than 8 knots. Wind Gusts MOS does not forecast wind gusts so MOS statistics are not available. A forecast 'hit' occurs if gusts are forecast and observed and the absolute difference between the TAF and MTR wind gust speed is less than 10 knots. All wind gusts less than 16 knots are ignored.
TEMPO Groups
The TEMPO group is defined in WSOM D-31 as "..shall be used to indicate temporary fluctuations to forecast meteorological conditions which are expected to: Have a high percentage (50% or greater) probability of occurrence, and, Last for one hour or less in each instance, and, In the aggregate, to cover less than half of the period..." This means conditions for a particular element should vary whenever a TEMPO group is used for that element. The inclusion of a TEMPO group in the forecast means two forecasts are valid at the same time. Refer to section 2 for a description of how Aviation Verify selects which forecast to verify. In addition to the standard TAF verification statistics, the following specialized statistics are computed for just TEMPO groups. These statistics are not stratified by projection (all 24 hours of the TAF are considered together). Each element (i.e., ceiling, visibility, weather, etc.) is tallied separately for every 5 minutes that the TEMPO group is valid. Hrs Number of hours (hrs) each element was forecast in TEMPO groups. GT Good Tempo (GT) is the percentage of time the element used in the TEMPO group varied enough to justify TEMPO usage. "Varied enough" for each element is defined in section 2, step 1. GT-HIT Good Tempo-Hit (GT-HIT) is the percentage of time... The element used in the TEMPO group varied enough to justify TEMPO usage (section 2, step 1), and The forecast element in the TEMPO group was a hit, i.e., matched the MTR observations. Ideally, this number would be around 50. Example: The observations indicated that ceilings varied sufficiently to justify a TEMPO group, the TAF prevailing group forecast ceilings at 800 feet, the TEMPO group forecast ceilings at 300 feet, and ceilings below 500 feet were observed 40% of the time. GT-HIT: 40. GT-IMP Good Tempo-Improved (GT-IMP) is the percentage of time... The element used in the TEMPO group varied enough to justify TEMPO usage (section 2, step 1), and The forecast in the TEMPO group improved upon the forecast in the prevailing group even though the forecast in the TEMPO group did not match the MTRs. Example: The observations indicated that ceilings varied enough to justify a TEMPO group, the TAF prevailing group forecast ceilings at 1200 feet, the TEMPO group forecast ceilings at 700 feet, ceilings below 500 feet were observed 40% of the time, and ceilings 1000 feet or higher were observed 60% of the time. GT-IMP: 40. Tempo S/B FM Given that the MTRs did not vary enough to justify a TEMPO group, this is the percentage of time the TEMPO group produced a correct forecast, while the prevailing group was incorrect. Example: The observations indicated that ceilings did not vary enough to justify a TEMPO group, the TAF prevailing group forecast ceilings at 1200 feet, the TEMPO group forecast ceilings at 800 feet, ceilings were observed below 500 feet 100% of the time. TEMPO S/B FM: 100. Tempo Benign Of all TEMPO hours for a given element, this the percentage of time... The TEMPO group produced a forecast more in error than the prevailing group, and The TEMPO group forecast better flying conditions than the prevailing group. Hence, the more erroneous TEMPO group was benign-it was unnecessary but also not harmful to the TAF. Example: The TAF prevailing group forecast ceilings at 700 feet, the TEMPO group forecast ceilings at 1200 feet, and ceilings were observed between 500 and 900 feet 90% of the time. Tempo Benign: 90. Tempo Hurt TAF Of all TEMPO hours for a given element, this the percentage of time... The TEMPO group produced a forecast more in error than the prevailing group, and The TEMPO group forecast worse flying conditions than the prevailing group. Hence, the more erroneous TEMPO group hurt the TAF operationally by forcing the pilot to plan for worse conditions that did not happen. Example: The TAF prevailing group forecast ceilings at 1400 feet, the TEMPO group forecast ceilings at 600 feet, and ceilings were observed between 1000 and 1900 feet 90% of the time. TEMPO Hurt TAF: 90.
PROB Groups
The PROB group is defined in WSOM D-31 as "...shall be used by NWS offices only to forecast a low probability occurrence (30 or 40 percent chance) of a thunderstorm (and associated precipitation) or precipitation event, along with associated weather and obscuration elements (wind, visibility and/or sky condition) whose occurrences are directly related to, and contemporaneous with, the thunderstorm or precipitation event." Whenever a PROB group is used, two forecasts are in effect. Based upon operational considerations, one of those two forecasts is selected for verification. If the condition in the PROB group is worse (more restrictive to flying) than the condition in the prevailing group, the PROB condition is verified, since that is the condition for which the pilot must plan. If the condition in the PROB group is better than the prevailing group, the prevailing condition is verified. In addition to the standard TAF verification statistics, the following specialized statistics are computed for just PROB groups. These statistics are not stratified by projection (all 24 hours of the TAF are considered together). Each element (i.e., ceiling, visibility, weather, etc.) is tallied separately for every 5 minutes that the PROB group is valid. The bold faced titles are the abbreviations used in the output. HRS Number of hours each element was forecast in PROB groups. %V % Verified (for weather element) is the percentage of time the weather element in the PROB group matched the weather element in the MTRs. The number of 5 minute segments when PROB weather actually occurred is divided by the number of 5 min segments that PROB weather was forecast. The fraction is than converted to a percentage. For example, a 3-hour PROB group for rain contained twenty four 5-minute segments (two hours) with rain observed: (24/36)100% = 66%. This statistic is computed separately for each of the following weather elements: thunderstorms, rain, snow, and freezing precipitation. Ideally, this number should be between 30 and 40. Note: This statistic is also computed for each non-weather element included in the PROB group. %V % Verified (for non-weather element) is the percentage of the time each non-weather element in the PROB group verified. VWX Given only the verified PROB groups for each weather element (thunderstorms, rain, snow, freezing precipitation), this is the percentage of time each non-weather element (ceiling, visibility, wind, wind gusts) in the forecast verified. Ideally, this score would always be 100. % Improved TAF % Improved TAF (+) is the percentage of time the non-weather elements in the PROB group resulted in a more accurate forecast than the prevailing group. For example, LIFR ceilings are observed, VFR ceilings are forecast in the prevailing group, and IFR ceilings are forecast in the PROB group. The PROB group improved the forecast. % Hurt TAF (-) % Hurt TAF (-) is the percentage of time... The non-weather elements in the PROB group resulted in a less accurate forecast than the prevailing group, and The PROB group forecast worse (lower) flying conditions than the prevailing group. For example: MVFR ceilings were observed, IFR ceilings were forecast in the prevailing group, and LIFR ceilings were forecast in the PROB group.
Verification Parameters
Period of Evaluation - Dates are given TAF Hour Forecaster Percentage of MTRs is the percentage of MTRs that were available for the TAFs that were evaluated. If two days worth of MTRs were missing, and the TAFs that referred to those MTRs were also missing, this statistic would not be affected. Percentage of TAFs is the percentage of possible TAFs that were verified for the requested period. Percentage of MOS is the percentage of possible MOS forecasts verified for the requested period.
If you have any questions about the documentation...please e-mail Charles Kluepfel or Michael Graf. If you have any problems with the program, please notify the Southern Region Aviation Mereorologist Paul Witsaman. A FAQ website is available based on questions from the field.
Overview | Downloads | Multiple Forecasts At One Time | Statistics | Output | Quick Set Up Printing problems | Transferring data | Aviation Home | SR Home