Description
Eco 519 Final: Empirical Project on Probability forecasts for binary events (20 points towards the Final)
I have attached data on a time series of probability forecasts for real GNP/GDP declines in the next quarter from 1968:4th quarter, when the Survey of Professional Forecasters (SPF) started. This data set ends at 2015Q4 in the Excel sheet. You have to update the data till 2019Q2. The probabilities are the averages over all respondents in the survey in each quarter. Use the link below to the SPF real GDP data page. On this page, towards the bottom, there is a link to “Data for error statistics,” which is the file that contains real time data vintages for the target variable.
The actual real GNP growth in the targeted quarter (real-time preliminary 1st-month announcements) and the corresponding (1, 0) dummy variable indicating 1 if the quarter experienced a negative real GNP decline are also provided. Since you will need this binary outcome variable in your calculations and since there hasn’t been any negative GDP growth since after 2015Q3, you can safely use 0 in the target variable for the rest of the sample period. Also, due to Government shutdown, real-time GDP growth has a missing value for 1995Q4. Since this quarter didn’t have negative growth, use “0” for this quarter also.
Based on these two series, do the following:
1) Compute the QPS.
2) Run an OLS regression of the dummy (1, 0) variable on the probability forecasts to test if these forecasts are unbiased.
3) Draw the two conditional distributions of the forecasts given the actual is 1 or 0, on the same diagram and note the probability over which these two lines intersect. You can smooth the curves. Do these conditional densities look good and useful? Which one looks better? What does this mean in terms of the effectiveness of these forecasts to guide people regarding an impending negative growth quarter?
4) Draw the Receiver Operating Characteristic (ROC) curve with values of the threshold ranging from 0 to 1 at an increment of 0.1. Below 0.5, you can search for optimal value for the threshold at increments of .05.
5) One popular criterion to choose the optimal threshold to convert probability forecasts into binary forecasts is Kuiper’s or Peirce Skill (PS) score. How is it defined? What threshold does PS suggest for these forecasts? Is it close to the probability over which the two conditional densities intersect (see #3 above)? What are the associated hit rate and false alarm rates for this optimal threshold? Are these values reasonable to you given your personal risk appetite for type I and type II errors in forecasting recessions?