FORECAST ERRORS IN PRICES AND WAGES : THE EXPERIENCE WITH THREE PROGRAMME COUNTRIES

This paper evaluates the accuracy of price and wage forecasts made under the scope of the Greek, Portuguese and Irish Economic Adjustment Programmes (EAPs). Besides comparing the quality of forecasts for the three EAPs, it checks if they were less accurate than forecasts for other euro area countries, and compares the European Commission’s forecasts with those of the IMF and OECD. Programme forecasts for Greece were, overall, the least accurate, but those made under the second Greek EAP seem to have gotten closer to actual values. The results also suggest that EC forecasts were not less accurate than those produced by the IMF or the OECD.


INTRODUCTION
Economic outlooks were central pieces of the Economic Adjustment Programmes (EAPs) implemented in European Union countries affected by the sovereign debt crisis.In fact, those economic forecasts were used as a basis for the design of the policy measures Greece, Ireland, Portugal, and Cyprus were asked to adopt.Given this central role of economic forecasts, it is vital that they are of high quality.Unfortunately, forecasts are seldom as accurate as economists and policymakers would desire, and forecast errors can potentially lead to inappropriate policies which generate significant economic costs.That may have been the case in Programme countries such as Greece and Portugal, where austerity measures were especially harsh.
This paper evaluates the forecast performance of the Greek, Irish and Portuguese Programmes in the specific area of prices and wage developments, using data between 2009 and 2014.First, the forecasted behaviour of wages and prices is compared with the values that actually materialized, checking if forecast errors were larger for Greece (where the greatest forecast errors seem to have been committed) 1 than for Portugal and Ireland, and/or for the EU countries in general, and comparing EC forecast errors with those of the IMF and the OECD.Developments in prices are proxied by the annual percentage change in the Harmonized Index of Consumer Prices (HICP), while developments in wages are proxied by annual percentage changes in unit labour costs (ULC). 2 The paper is structured as follows.Section 2 provides a graphical presentation of the forecasts and forecast errors, regarding prices and wages, for the Greek, Irish and Portuguese EAPs.Section 3 compares the forecast accuracy of the Programmes' forecasts using measures such as the mean error (ME), the mean absolute error (MAE), and the root mean squared error (RMSE).The following exercise, in Section 4, compares the current year, 1-year ahead and 2-years-ahead forecasts of the EC with those of other international agencies, such as the IMF and the OECD. 3 In Section 5, econometric estimations are performed in order to check if error statistics for the 3 programme countries are higher than for other euro area countries.Section 6 concludes the paper.

PROGRAMME FORECASTS OF PRICE AND WAGE DEVELOPMENTS
This section presents the successive programmes' forecasts for price and wage developments, until April 2014, and compares them with actual values, taken from the February 2015 release of the European Commission's AMECO database.

GREECE
The successive inflation forecasts included in the Greek EAPs are presented in Figure 1.A vertical bar is included for the initial forecasts and for those of each of the periodic reviews, while actual values are shown in a black line and with numbers next to it.The initial inflation forecasts (in May 2010) were generally far from the actual values.Forecasts for 2010 and 2011 were well below the values that materialized, while those of 2013 and 2014 were above them.Only the initial forecast for 2012 (1.2%) was close to the actual value (1%).Subsequent forecasts made in 2010 corrected the estimate for that year, but underestimated inflation in 2011 and 2012 and overestimated it in 2013 and 2014.In fact, practically all forecasts overestimated inflation for these latter years.
Inflation forecast errors for the current year, 1-year-ahead and 2-years-ahead are presented in Figure 2.These errors were calculated as the difference between each forecast and the corresponding value obtained from the February 2015 release of AMECO. 4 As mentioned above, the initial programme forecasts (May 2010) were quite far from the final values.Except for the latter reviews, inflation tended to be underestimated in the forecasts for the current year and 1-year ahead.4 shows that forecast errors for the current year were mostly negative (except for the last 3 reviews), while for 1-year-ahead and 2-years-ahead were mostly positive (again, with the exception of the last 3 reviews).Forecast errors for the annual changes in unit labour costs (ULC) are illustrated in Figure 6.Forecasts made in 2011 for the current year and 1-year-ahead overestimated the growth in ULC, while latter forecasts underestimated it.Two-years-ahead forecasts generally underestimated the change in ULC.Although errors are generally higher for ULC than for  Julho '17 (7-24)   12 inflation, the same happened for Greece, which had even higher forecast errors in ULC than Portugal, in some occasions exceeding 6 percentage points (see Figure 4).

IRELAND
A pattern similar to that of inflation forecasts for Portugal is observed for Ireland in Figure 7, as initial current year and 1-year-ahead forecasts tended to underestimate inflation, while latter ones overestimated it.As for Portugal, 2-years-ahead forecasts always overestimated inflation, and the biggest forecast errors were around 1 percentage point.Forecast errors in changes in ULC for Ireland, shown in Figure 8, were generally higher than those for Portugal, and comparable to those for Greece.In fact, changes in ULC were sometimes underestimated by almost 6 percentage points.

FORECAST PERFORMANCE OF THE THREE ECONOMIC ADJUSTMENT PROGRAMMES
This section compares the forecast performance of the three EAPs (Greece, Portugal, and Ireland), using three summary statistics: the mean error (ME), the mean absolute error (MAE), and the root mean squared error (RMSE).5

FORECAST ERRORS AND SUMMARY STATISTICS
Following Cabanillas and Terzi (2012), and previous studies, we define the forecast errors and the summary statistics as explained below.
The forecast error is the difference between the forecast and the realization.That is: , for the current year , for the year ahead , for two years ahead, where y t,t , y t+1 ,t and y t+2,t are the forecasts of y made at period t for t, t+1 and t+2, respectively; and y t , y t+1 and y t+2 are the realizations of y in years t, t+1 and t+2, respectively.The programmes' forecast accuracy is assessed using three commonly used summary statistics: The mean error (ME), which is the average difference between the forecast and the outturn.Although it has the problem that negative errors may compensate positive ones, not being a good accuracy indicator, it may help signal biases in the forecasts.
Table 1 shows the summary statistics for the inflation forecasts made in the EAPs of Greece, Portugal, and Ireland.As in the figures shown above, all available programme forecasts (initial and of all reviews) for each country were considered.
Regarding current year forecasts, the mean error for Greece is negative, indicating that forecasts, on average, underestimated inflation in that country.The positive ME in Portugal, which is close to the MAE, indicates that inflation forecasts for Portugal tended to overestimate actual inflation.For Ireland, the ME is much smaller than the MAE, which indicates that positive errors were in large part cancelled out by negative ones.The MAE is higher for Greece than for the other countries, although the difference to Ireland is not very big.But, the RMSE for Greece is twice as big as that for Ireland (the second largest), which is due to the large errors made in the initial inflation forecasts for Greece.Although the 1-year-ahead forecasts have a somewhat similar pattern (especially for the ME and RMSE), those of 2-years-ahead are quite different, with Portugal exhibiting the largest ME, MAE and RMSE.Thus, it is interesting to note that forecast accuracy for Portugal was the highest when considering the current year, but it was the lowest for 2-years-ahead.For 1-year-ahead, Greece has the lowest accuracy according to the RMSE (which penalizes the large errors of the initial programme forecasts), and the highest according to the MAE.
Table 2 presents the summary statistics for the forecasts of changes in unit labour costs (ULC).Regarding current year forecasts, the lowest accuracy was achieved for Ireland, according to any of the measures.The same applies to 1-year-ahead forecasts, according to the MAE and RMSE.As for 2-years-ahead forecasts, the lowest accuracy was for Greece, according to all statistics.Additionally, forecasts for Greece were always less accurate than those for Portugal.In sum, there is some indication that forecasts for Greece were less accurate than for Portugal and Ireland regarding inflation (for current year and 1-year-ahead forecasts).Regarding ULC forecasts, those for Greece were still less accurate than for Portugal, but more accurate than those for Ireland.

COMPARISON OF EC FORECAST ERRORS WITH THOSE OF THE IMF AND OECD
In order to check if the European Commission's forecasts were more or less accurate than those of the IMF and OECD, we show in this section the summary statistics of those institutions' forecasts.Following Cabanillas and Terzi (2012), current-year forecasts are taken from the Commission's spring forecasts, whereas 1-year-ahead forecasts come from the Commission's autumn forecasts.The 2-years-ahead forecasts, not used in that study, are also taken from the autumn forecasts.The same procedure is used regarding the IMF forecasts, taken from its World Economic Outlook, and the OECD forecasts, taken from its Economic Outlook. 6The first forecast considered for each institution is that of autumn 2009 and the last is that of autumn 2014.We start by comparing forecasts for Greece, Portugal and Ireland, and then we show Ordinary least Squares (OLS) regression results for estimations including other euro area countries.

FORECASTS STATISTICS FOR GREECE, PORTUGAL AND IRELAND
The forecast summary statistics for the inflation forecasts of the EC, IMF and OECD, for Greece, Portugal and Ireland, are shown in Table 3. IMF forecasts were generally the least accurate, according to the MAE and RMSE, regardless of the country and time horizon considered.7 Regarding current year forecasts, the OECD performed somewhat better than the EC for Greece and Ireland, while their accuracy was practically the same for Portugal.The EC's 1-year-ahead forecasts were generally more accurate than those of the OECD for Greece, slightly better for Ireland, and about the same for Portugal (the EC has a lower MAE, but a higher RMSE).Regarding 2-years-ahead forecasts, the EC performed worse than the OECD for Greece and Ireland, but better for Portugal.
Table 4 shows the summary statistics for EC and OECD forecasts of annual percentage changes in unit labour costs. 8The EC forecasts for Greece were more accurate than the OECD's when considering the current year and the year ahead, but the opposite happened for 2-years-ahead.Regarding forecasts of changes in ULC for Ireland, the OECD was always more accurate than the EC.A comparison is not possible for Portugal, as there were no ULC forecasts available for this country in the OECD's Economic Outlook.
In sum, it is not possible to state that the European Commission's forecasts were always the most accurate.But, it is safe to argue that the EC's inflation forecasts for the 3 programme countries considered were always more accurate than those of the IMF.9 Additionally, the EC forecasts for inflation and ULC do not seem to be, on average, less accurate than those of the OECD.

COMPARISON OF FORECAST ERRORS FOR PROGRAMME COUNTRIES WITH THOSE FOR OHER EURO AREA COUNTRIES
In order to check if the forecast accuracy for the three programme countries analysed above was different from that for other 14 euro area countries,10 we estimated individual OLS cross-country regressions for each variable (inflation and ULC), institution (EC, IMF and OECD), and summary statistic (ME, MAE and RMSE).In each regression, the dependent variable is a summary statistic, and the explanatory variables are the dummy variables for the three programme countries and a constant.The estimated model is the following: , where FS i is a forecast statistic (ME, MAE or RMSE) for country i; Greece i , Portugal i and Ireland i are dummy variables that take the value of one for the respective countries and zero otherwise; is a white noise error term; and , and are parameters to be estimated.
The results of the regressions for inflation forecast statistics are presented in Table 5.11 Regarding current year forecasts, the dummy variable for Greece is positive and statistically significant for the MAE and RMSE, which indicates that EC inflation forecasts for Greece were less accurate than for the remaining euro area countries.The same applies to the IMF and the OECD.There is also some indication that IMF and OECD forecasts for Ireland were also less accurate than for the other euro area countries (but more accurate than those for Greece).Finally, the regression for the MAE, indicates that OECD inflation forecasts for Portugal may have been more accurate than for the other euro area countries.
A different picture emerges for the 1-year-ahead forecasts, as EC forecasts for the three programme countries seem to have been more accurate than for the remaining euro area countries.This is not true for IMF forecasts, since there is a positive and statistically significant coefficient for Greece in the RMSE regression, and the dummy variables for Portugal and Ireland are not statistically significant.As for OECD year ahead forecasts, there is weak evidence of higher errors for Greece (for the RMSE) and lower ones for Portugal and Ireland, than for the remaining euro area countries.
Regarding the 2-years-ahead forecasts of the EC, there is no robust evidence of different forecast accuracy for the programme countries.For the IMF, there is indication of higher errors for Ireland and lower ones for Portugal.Finally, the OECD 2-years-ahead forecasts seem to have been more accurate for Greece and Portugal.

Forecast Errors in Prices and
Wages: The Experience with Three Programme Countries Table 6 presents the results of regressions for the forecasts of changes in unit labour costs (ULC) by the European Commission and the OECD.Current year ULC forecasts for the programme countries seem to have been less accurate than for the other euro area countries.In fact, all dummy variables for programme countries are positive and statistically significant in the regressions for the MAE and the RMSE.The same applies to 1-year-ahead forecasts of the EC for Portugal and of the OECD for Greece and Ireland.Regarding 2-years-ahead forecasts, they seem to have been less accurate for Greece and Portugal than for the remaining euro area countries.Thus, overall, ULC forecasts for the 3 programme countries tended to be less accurate than for the other euro area members.

COMPARISON OF FORECAST ERRORS COMBINING THE THREE INSTITUTIONS
A final econometric analysis consisted on estimating regressions for a sample which combines the forecast statistics of the 3 institutions.In order to check if the EC's forecasts were in general more or less accurate than those of the IMF and OECD, dummy variables for those institutions were added to the list of explanatory variables.The results are presented in Table 7.In the regressions for current year inflation forecasts, the dummy variable for the IMF is positive, while that for the OECD is negative.Since both are statistically significant, this indicates that EC inflation forecasts were, on average, more accurate than those of the IMF, but less accurate than those of the OECD.As for ULC, the OECD dummy variable is not statistically significant, indicating that there was no difference in accuracy relative to EC forecasts.MAE regressions indicate that EC inflation forecasts 1 and 2-years-ahead may have been more accurate than those of the IMF and OECD.For ULC, there is no difference in 1-year-ahead forecasts, but the EC seems to have been more accurate than the OECD in 2-years-ahead forecasts (in the MAE estimation).
The results for the country dummy variables indicate that inflation current year forecasts were, on average (and considering the 3 institutions), less accurate for Greece than for the remaining euro area countries.As for ULC forecasts (by the EC and OECD), they were less accurate for the 3 programme countries, especially for Greece and Portugal, than for the others.For 1-year-ahead forecasts, the MAE regression indicates that greater than average accuracy may have been achieved in inflation forecasts for Ireland, while there was lower than average accuracy in the 1-year-ahead ULC forecasts for Portugal.Finally, concerning 2-years-ahead, inflation forecasts for Portugal were more accurate and those for Ireland were less accurate than average.ULC forecasts were less accurate for Greece than for the other countries.

CONCLUSIONS
The analysis presented in this study leads us to conclude that programme forecasts for Greece were less accurate than for Portugal and Ireland regarding inflation.Initial inflation forecasts (in May 2010) for Greece, for the current year and the year ahead, were far below from the actual values.The same, although to a smaller degree, happened with the initial forecasts of the second adjustment programme for Greece.Forecasts for unit labour costs (ULC) were less accurate for Ireland regarding the current year and year ahead, and less accurate for Greece for 2-years-ahead.Comparing to ULC programme forecasts for Portugal, forecasts for Greece were always less accurate.Although programme forecasts for Greece were, overall, the least accurate of the 3 programmes' forecasts, those made in more recent reviews of the Greek EAP seem to have gotten closer to actual values.
The forecasts errors in programme countries may in part have been due to greater difficulty to forecast in more volatile environments.In fact, IMF and OECD forecasts for these three programme countries also had relatively large errors, with IMF inflation forecasts being less accurate than those of the European Commission for any of the three countries.12Additionally, current year inflation and ULC forecasts for Greece were generally less accurate than those for the other euro area countries, regardless if they were made by the EC, IMF, or OECD.Actually, ULC forecasts for Greece, Ireland and Portugal were always less accurate than for the other euro area members.
Overall, EC forecast do not seem to be less accurate than those of the IMF and OECD.In fact, regarding inflation, they are on average more accurate than those of the IMF.

Figure 1 :
Figure 1: Inflation forecasts for Greece

Figure 2 :
Figure 2: Inflation forecast errors for Greece Figure 3 presents the forecasts and actual values of annual percentage changes in unit labour costs (ULC).As happened with inflation, the initial programme forecasts were below the actual values of 2010 and 2011 and above actual values in 2013 and 2014.Figure4shows that forecast errors for the current year were mostly negative (except for the last 3 reviews), while for 1-year-ahead and 2-years-ahead were mostly positive (again, with the exception of the last 3 reviews).

Figure 3 :
Figure 3: Unit Labour Costs forecasts for Greece

Figure 5 :
Figure 5: Inflation forecast errors for Portugal

Figure 4 :
Figure 4: Unit Labour Costs forecast errors for Greece

Figure 6 :
Figure 6: Unit Labour Costs forecast errors for Portugal

Figure 7 :
Figure 7: Inflation forecast errors for Ireland

Figure 8 :
Figure 8: Unit Labour Costs forecast errors for Ireland

Table 1 :
Programme Inflation forecasts statistics Sources: European Commission, European Economy, Occasional Papers (several issues) and AMECO.

Table 2 :
Programme ULC forecasts statistics Sources: European Commission, European Economy, Occasional Papers (several issues) and AMECO.

Table 3 :
Inflation forecast statistics Sources: European Commission, European Economy, Economic Forecast and AMECO; International Monetary Fund, World Economic Outlook; and OECD, Economic Outlook.Sources: European Commission, European Economy, Economic Forecast and AMECO; and OECD, Economic Outlook.

Table 5 :
Regressions for inflation forecasts Sources: European Commission, European Economy, Economic Forecast and AMECO; IMF, World Economic Outlook; and OECD, Economic Outlook.Notes: OLS models estimated for a sample of 17 euro area countries, from autumn 2009 to autumn 2014.The estimated coefficient for the constant is not shown.T-statistics are in parentheses.Significance level at which the null hypothesis is rejected: ***, 1%; **, 5%, and *, 10%.

Table 6 :
Regressions for forecasts of annual changes in Unit Labour Costs (ULC)

Table 7 :
Regressions forecasts combining the three institutions OLS models estimated for a sample of 17 euro area countries, from autumn 2009 to autumn 2014.There are 49 observations for inflation and 31 for ULC.The estimated coefficient for the constant is not shown.T-statistics are in parenthesis.Significance level at which the null hypothesis is rejected: ***, 1%; **, 5%, and *, 10%.
Sources: European Commission, European Economy, Economic Forecast and AMECO; IMF, World Economic Outlook; and OECD, Economic Outlook.