How do WSJ survey forecasters measure up? – Research Notes & News
Twice a year the Wall Street Journal surveys a group of forecasters for their forecasts of several key macroeconomic variables designed to characterize the economy’s future performance. The Journal publishes the forecasts and provides a ranking of a few of the top forecasters based on how close the forecasts of the variables are to their realized values.
The methodology used to rank the forecasters scores them on the sum of the weighted absolute percentage deviation from the actual realized value of each series; the weight for each series is simply the inverse of the actual realized value of the series. This performance-assessment method may become distorted, and even undefined, when the realized value is close to or equal to zero. Also, it does not consider the correlations in the data among the variables being forecast.
In a recent working paper, Robert Eisenbeis, Daniel Waggoner and Tao Zha propose a methodology that is designed to be used to assess the accuracy of any type of forecast and that both yields a measure of joint forecast performance and provides a single measure of how similar a joint forecast is to those of other forecasts. The method also allows the authors to assess the collective forecast accuracy of a set of forecasts flowing from economic models or individual forecasters and the accuracy of those forecasts over time. Finally, the method provides some indication of how tightly the forecasts are clustered around the realized values and can be used to compare judgmental forecasts as well as those of formal econometric models.
Among many results, the authors find great variability among forecaster performance over time, which serves to illustrate how difficult economic forecasting is.
WORKING PAPER 2002-8A
JULY 2002
COPYRIGHT 2002 www.frbatlanta.org
COPYRIGHT 2004 Gale Group