Germán Villar
1.0
2015-06-14
Loss functions to evaluate out of sample predictions
Germán Villar Argueta 1.0 2015-05-15 Loss_Functions This function returns different criteria to evaluate out of sample predictions. Also, there are different types of loss functions to evaluate predictions, given the type of prediction (forecast, asymmetry, probability, interval, etc.). The syntax is the following: Loss_Functions(h, y, yhat, p, d, a), where ‘h’ is forecast horizon; ‘y’ is the observed variable; ‘yhat’ is the predicted variable;’p’ is the probability of occurrence of a turning point at date t;’d’ is a dummy equal to 1 if there is a business cycle turning point (or any defined event) in period t and 0, otherwise; and ‘a’ is a scalar with a value less than 1/2 associated to the linex function defined in Hwang, Knight & Satchell (1999). Finally, this function returns three types of loss functions arranged in three vectors inside a bundle: Financial Forecast Loss Functions (FFLF), Asymetry Linex Loss Functions (ALLF), and Probability Forecast Loss Functions (PFLF). The first vector, defined as ‘Lfor’, gives us six different FFLF’s defined in Lee(2007); the second vector returns four different ALLF’s defined in Hwang, Knight & Satchel (1999); the third vector returns two PFLF’S defined in Lee(2007).
--------------–
References
Hwang, S., Knight, J. L., & Satchell, S. (1999). Forecasting volatility using LINEX loss functions.
Lee T. H., (2007): “Loss Functions in Time series forecasting”, University of California, Riverside, Unpublished working paper
prediction horizon
observed variable
prediction of y
pdf
dummy variable
a<1/2
```
d1 = (yhat/abs(yhat))*(y/abs(y))
genr dum = (d1>0)
#Loss functions for Financial Forecasts
genr MSFEsq = (sum(y*y) + sum(yhat*yhat) -2*sum(y*yhat))/h
genr MSFE = MSFEsq^(1/2)
genr MAFE = sum(abs(yhat-y))/h
genr MAPE = sum(abs((yhat-y)/y))/h
genr MFTR = -sum((yhat/abs(yhat))*y)/h
genr MCFD = -sum(dum)/h
#Loss functions for assymetry (Linex Functions)
genr LCF = ln(2) - ln((gamma(.5-a))/gamma(.5))/a
genr MAFElinex = sum(abs(abs(y)-(yhat*exp(LCF))^(1/2)))/h
genr MSFElinex = sum((abs(y)-(yhat*exp(LCF))^(1/2))^2)/h
genr MALFElinex = sum(abs(ln(y^2)-ln(yhat)-LCF))/h
genr MSLFElinex = sum(abs(ln(y^2)-ln(yhat)-LCF)^2)/h
#Loss functions for probability forecasts
genr QPS = sum(2*(p-d)^2)/h
genr LPS = -sum(ln((p^(d))*(1-p)^(1-d)))/h
matrix LFOR = {MSFEsq, MSFE, MAFE, MAPE, MFTR, MCFD}
colnames(LFOR, "MSFEsq MSFE MAFE MAPE MFTR MCFD")
matrix LASSYM = {MAFElinex, MSFElinex, MALFElinex, MSLFElinex}
colnames(LASSYM, "MAFElinex MSFElinex MALFElinex MSLFElinex")
matrix LPF = {QPS, LPS}
colnames(LPF, "QPS LPS")
bundle LF
LF["LF_for_Financial_Forecasts"] = LFOR
LF["LF_for_assymetry"] = LASSYM
LF["LF_for_probability_forecasts"] = LPF
print LF
print LFOR
print LASSYM
print LPF
return LF
```

include loss_functions.gfn
open arma.gdt
d = (u<0)
bundle evaluate = loss_functions(100, y, u, eps, d, .2)
#'a' is a scalar such that a<.5. See Hwang, S., Knight, J. L., & Satchell, S. (1999).
print evaluate