The Cold, Dry and Bitter Truth about Seasonal Forecasts

“Some suffer from an acute expert problem, producing cosmetic but fake knowledge, particularly in narrative disciplines…” 

~ Nassim Taleb, The Black Swan.

Periodic headlines scream ‘Environment Canada Blows Winter Forecast’. The next day ‘Environment Canada Forecasts Warm Spring Ahead...’ The following day it’s ‘Groundhog Prophecies Mixed on spring’s Arrival’. The media are stuck in a loop, rushing headlong to create the seasonal weather story every three months, overselling it as Weather You Should Fear Today! In the rapid-fire nothing-is-too-inconsequential-to-be-made-consequential twitter-verse, the next weather horror is never far away. Once established, the hair-brained scare is impossible to unseat. In 2010, it was ‘The Worst Winter in 50 Years’. In 2011, it was ‘the Coldest Winter in 20 Years’. The oversell nicely sets up the inevitable end-of-season follow-up story about the ‘blowing’ of it. It’s really much ado about nothing and enough to make a meteorologist moan.

In my 30+ year career, I’ve been asked for the seasonal ‘outlook’ more than any other forecast. The question wrongly presumes:

a)     an answer is possible

b)     such a forecast exists

c)     a meteorologist can provide it

The cold, dry and bitter truth is:

1) a useful answer is impossible

2) no such forecast exists and,

3) you’re asking the wrong expert anyway

1)  A useful answer is impossible because a highly variable element cannot be described by its average. For example: the mean temperature of a fall day that starts frosty at 0° and peaks at 20° in glorious sunshine is identical to that of a rainy day where the temperature never budges from 10°.

Tmn frosty sunshine day = (0°+ 20°)/2   = 10°          

Tmn rainy day = (10°+ 10°)/2   = 10°

                                      

This averaging is a ‘first order of smoothing’ that severely constrains our ability to describe the day’s weather from the temperature alone. Now, consider that the seasonal temperature forecast is the average of 90 days of mean temperatures. Or, working in reverse: ask yourself what might be said about today’s weather in any city when given the average of today’s mean temperatures from 90 cities? Answer: absolutely nothing and likewise for the average of 90 days worth of forecast temperatures for a single city. The ‘average of averages’ is a second order of smoothing that yields a result devoid of useful information.

2) Believe it or not, there is no such thing as a ‘seasonal weather forecast’. Weather – meteorology - is simply not predictable beyond a week at best. Predictions of ninety-day average temperature and total precipitation – climatology - are made by the climatology divisions of various national and international agencies. Short of a grocery-stand almanac – whose forecasts come from ‘a secret formula that was devised in 1792 and that remains locked in a black box in a New Hampshire office’, no credible agency even attempts to produce a weather forecast to day-90. Furthermore, candid climatologists concur; verification demonstrates very little ‘skill’ in 90-day temperature outlooks and practically zero skill in the precipitation outlooks.

The sole exception to death-by-smoothing is the case of an extreme season where the mean temperature or the total precipitation may hint at the weather experienced. This however, only applies looking backward at what has happened, rather than looking forward to what may happen because of a third order of smoothing essential to the forecast strategy: Seasonal outlooks are merely predictions of the broad ranges into which the 90-day mean temperature or the total precipitation is expected to fall: below normal, near normal or above normal. With only three possible outcomes, these predictions – by design - simply cannot identify or ‘capture’ extreme seasons. The ranges are broad for good reason: to quote Nils Bohr, “prediction is very difficult, especially about the future”. A consequence of this scheme is that even a perfect forecast won’t discern anything about the weather. Predictions of 90-day averages and totals may be valuable to climatologists or to utilities that need to hang their hats on something to guess at seasonal energy consumption, but they have zero value to the public trying to extract weather forecasts.

3) You’re asking the wrong expert anyway. Meteorologists have no training, experience or skill at forecasting beyond five days. The 2011 worst-winter-in-20-years story was hatched by the 'Canadian-weather expert' at a private company in the United States. To find out how this forecast was created, I emailed the expert a few times.  After four weeks without a response, I left a voice-mail posing as a reporter. Within an hour I had a call-back.

Posing Reporter: Are you a climatologist? 

Expert: No

Posing Reporter: Do you have any training in synoptic climatology? 

Expert: No

Posing Reporter: Are there any synoptic climatologists working for your company?  

Expert: No

Posing Reporter: Are these forecasts based upon climate models?  

Expert: No

Posing Reporter: Do you know the accuracy of your previous forecasts? 

Expert: No

Posing Reporter: How did you arrive at the ‘third coldest winter’ prediction?’

Expert: ‘Well, I just sorta looked at about the last twenty winters in Vancouver and it wouldn’t take much more than a degree or two colder conditions to put this into the top three coldest’. 

The 2011 winter narrative - spread like wild-fire across all major media - was based on mere speculation by a person with no climatological credibility. Most meteorologists don’t hesitate to defer questions about flooding, avalanches and meteors to hydrologists, avalanche forecasters, and astronomers. I’ll let you decide if this expert’s goal was to inform the audience or to gain an audience.

To quote Nassim Taleb again:

“At the core of the expert problem is that people are suckers for charlatans, particularly when the charlatan is invested with some institutional authority… they serve as experts while offering the scientific reliability of astrologers. Anyone relying on them is a turkey.’’

Don’t be a turkey.