Faculty of Management Sciences

Forecasting and Data Analysis (FDA621S)- forecast errors

Lecturer: Mrs. Emilia Salomo
esalomo@[Link]
Office # 130, Red floor, Office Building

12 September 2019
 FORECAST ACCURACY
Faculty of Management Sciences

Forecast accuracy is a significant factor when deciding among forecasting alternatives. Accuracy is based
on the historical error performance of a forecast. Three commonly used measures for summarizing
historical errors are the;
However, below two measures are also important when
calculating forecasting errors.
• Mean absolute deviation (MAD)
The average absolute forecast error. • Cumulative sum of forecast errors (CFE):
• Mean squared error (MSE) Total of all errors
The average of squared forecast errors. • Tracking signal
• Mean absolute percent error (MAPE)
Ratio of bias and MAD
The average absolute percent error.
 Measures of Forecast
Faculty of Management Sciences Error
Measure Description
Error Forecast – Actual Demand
Absolute Error Absolute deviation (positive error)
Mean Squared Error (MSE) Squared deviation of forecast from demand
Mean Absolute Deviation (MAD) Absolute deviation of forecast from demand
Mean Absolute Percentage Error (MAPE) Absolute deviation of forecast from demand as a
percentage of the demand

Cumulative sum of forecast errors (CFE): Some of all errors

Tracking signal (TS) Ratio of bias and MAD
 Forecast Error (=E)
Faculty of Management Sciences

Et = Ft – Dt

• Measures the difference between the forecast and the actual demand in
period t
• Want error to be relatively small
 Forecast Error- Absolute deviation -|Et|
Faculty of Management Sciences

At = |Et|

• Measures the absolute value of error in period t

• Want absolute deviation to be relatively small
 Forecast Error-
Faculty of Management Sciences CFE

CFEt = ∑nt=1 Et

• A measurement of the total forecast error that assesses the bias in the forecast
error
• Want bias to be as close to zero as possible
• A large positive (negative) bias means that the forecast is overshooting (undershooting) the
actual observations
• Zero bias does not imply that the forecast is perfect (no error) -- only that the mean of the
forecast is “on target”
 Mean absolute deviation
Faculty of Management Sciences (MAD)

MADn =1 ∑t=1
n
At
n
• Measures absolute error
• Positive and negative errors do not cancel out (as with bias)
• Want MAD to be as small as possible
• No way to know if MAD error is large or small in relation to the actual data
 Mean squared error
Faculty of Management Sciences (MSE)

1
MSEn =
n ∑nt=1 Et2

• Measures squared forecast error

• A measurement of the dispersion of forecast errors.
• Recognizes that large errors are disproportionately more “expensive” than
small errors
• Not as easily interpreted as MAD, MAPE -- not as intuitive
 Mean absolute percentage
Faculty of Management Sciences error (MAPE)

Et
∑n
t=1 100
MAPEn = Dt
n
• Same as MAD, except ...
• A measurement that relates the forecast error to the level of demand and is useful for
putting forecast performance in the proper perspective.
• Measures absolute deviation as a percentage of actual demand
• Want MAPE to be less than 10 (though values under 30 are common)
 Tracking signal (TS)
Faculty of Management Sciences

TSt = biast / MADt

• A measure that indicates whether a method of forecasting is accurately predicting

actual changes in demand.
• Want tracking signal to stay within (–6, +6)
• If at any period the tracking signal is outside the range (–6, 6) then the forecast is biased
 Summary of Forecast Error
formulas
Faculty of Management Sciences

• Cumulative sum of forecast errors (CFE): CFE = Et

Et2
• Mean squared error (MSE): MSE = n

• Mean absolute deviation (MAD): MAD =

|Et |
n
[ |Et | / Dt ](100)
• Mean absolute percent error (MAPE): MAPE = n
CFE
• Tracking signal TS = MAD
 CLASS ACTIVITY

Faculty of Management Sciences For below set of data calculate;

month Actual Forecast 3 month a) Forecast for October 2018
α= 0.10 moving using exponential Smoothing
average
[8)
Mar-18 38 42   b) 3 month moving average [6]
Apr-18 45   c) CFE [2]
May-18 44   d) TS [2]
e) MAD [2]
Jun-18 45  
f) MSE [4]
Jul-18 44   g) MAPE [4]
Aug-18 46    
Sep-18 38  
Oct-18 42  
Nov-18      
 CLASS ACTIVITY- SOLUTION

Faculty of Management Sciences

month Actual Forecast 3 month moving average (A-F) ERROR |Error| Error2 [|
α= 0.10 Error|/Actual
]x100

18-Mar 38 42   (4.00) 4.00 16 11%

18-Apr 45 41.60   3.40 3.40 11.56 8%
18-May 44 41.94   2.06 2.06 4.24 5%
18-Jun 45 42.15 42.33 2.85 2.85 8.15 6%
18-Jul 44 42.43 44.67 1.57 1.57 2.46 4%
18-Aug 46 42.59 44.33 3.41 3.41 11.64 7%
18-Sep 38 42.93 45.00 (4.93) 4.93 24.30 13%
18-Oct 42 42.44 42.67 (0.44) 0.44 0.19 1%
18-Nov   42.39 42.00
TOTAL 3.93 22.66 78.55 54%
N=8 N=8 N=8
CFE TS MAD MSE MAPE
3.93 1.39 2.83 9.82 6.76%
 Time Series Problem
Faculty of Management Sciences

• Determine forecast for periods 7 & 8 Period Actual

• 2-period moving average 1 300
• 4-period moving average 2 315
• 2-period weighted moving average 3 290
with t-1 weighted 0.6 and t-2 4 345
weighted 0.4 5 320
• Exponential smoothing with 6 360
alpha=0.2 and the period 6 forecast
7 375
being 375
8
14
 Faculty of Management Sciences

[|
Period Actual Forecast α= Error|/Actual
0.20 (A-F) ERROR |Error| Error2 ]x100
1 300 310- 10.00 10.00 100.00 3%
2 315 308.00 7.00 7.00 49.00 2%
3 290 309.40 - 19.40 19.40 376.36 7%
4 345 305.52 39.48 39.48 1,558.67 11%
5 320 313.42 6.58 6.58 43.35 2%
6 360 314.73 45.27 45.27 2,049.12 13%
7 375 323.79 51.21 51.21 2,622.85 14%
8   334.03
total 120.14 178.94 6,799.35 52%
CFE TS MAD MSE MAPE
120.14 4.70 25.56 971.34 7%
 Summary
Faculty of Management Sciences

1. What information does the bias and TS provide to a manager?

• The bias and TS are used to estimate if the forecast consistently over- or under forecasts
2. What information does the MSE and MAD provide to a manager?
• MSE estimates the variance of the forecast error
• VAR(Forecast Error) = MSEn
• MAD estimates the standard deviation of the forecast error
• STDEV(Forecast Error) = 1.25 MADn
 Forecast Error in Excel
Faculty of Management Sciences

• Calculate absolute error At

=ABS(Et)
• Calculate mean absolute deviation MADn
=SUM(A1:An)/n
=AVERAGE(A1:An)
• Calculate mean absolute percentage error MAPEn
=AVERAGE(…)
• Calculate tracking signal TSt
=biast / MADt
• Calculate mean squared error MSEn
=SUMSQ(E1:En)/n
 Forecast Error in Excel
Faculty of Management Sciences

Error

E_t
=C4-B4 Et = Ft – Dt
=C5-B5
=C6-B6
=C7-B7 Forecast Error
 Forecast Error in Excel
Faculty of Management Sciences

Bias

bias_t
=SUM($D$4:D4) biasn = ∑nt=1 Et
=SUM($D$4:D5)
=SUM($D$4:D6)
=SUM($D$4:D7) Bias
 Forecast Error in Excel
Faculty of Management Sciences

Absolute
Error
A_t
=ABS(D4) At = |Et|
=ABS(D5)
=ABS(D6)
=ABS(D7) Absolute Error
 Forecast Error in
Faculty of Management Sciences Excel

Mean
Abs Error
MAD_t
=AVERAGE($F$4:F4) MADn =1 ∑t=1
n A
t
n
=AVERAGE($F$4:F5)
=AVERAGE($F$4:F6)
=AVERAGE($F$4:F7) Mean Absolute Deviation
 Forecast Error in Excel
Faculty of Management Sciences

Tracking
Signal
TS_t
=E4/G4 TSt = biast / MADt
=E5/G5
=E6/G6
=E7/G7 Tracking Signal
 Forecast Error in
Faculty of Management Sciences Excel

|%Error|

|%Error| Et
=ABS(D4/B4)*100 |%Error|t = 100
Dt
=ABS(D5/B5)*100
=ABS(D6/B6)*100
=ABS(D7/B7)*100 |%Error|
 Forecast Error
Faculty of Management Sciences in Excel

Mean
|%Error|
MAPE_t ∑nt=1 |%Error|t
=AVERAGE($I$4:I4) MAPEn =
n
=AVERAGE($I$4:I5)
=AVERAGE($I$4:I6)
=AVERAGE($I$4:I7) Mean Absolute
Percentage Error
 Forecast Error in
Faculty of Management Sciences
Excel

Mean
Sq Error
MSE_t 1
MSEn = ∑nt=1 Et2
=SUMSQ($D$4:D4)/A4 n
=SUMSQ($D$4:D5)/A5
=SUMSQ($D$4:D6)/A6
=SUMSQ($D$4:D7)/A7 Mean Squared Error