holdout sample Archives - KDD Analytics

Practical Time Series Forecasting – Know When to Roll ‘em

By KDD | January 29, 2018

times series forecasting rolling holdout samples

“Prediction is very difficult, especially if it’s about the future.” ― Niels Bohr, physicist Holdout samples are a key component to estimating a “useful” forecasting model. Set aside data at least equal in length to your forecast horizon (“holdout sample”). Build your models on the remaining data (“modeling sample”). And compare the candidate models’ forecast…

Practical Time Series Forecasting – Know When to Hold ‘em

By KDD | January 8, 2018

times series forecasting holdout samples

“The only relevant test of the validity of a hypothesis is comparison of prediction with experience.” ― Milton Friedman, economist Holdout samples are a mainstay of predictive analytics. Set aside a portion of your data (say, 30%). Build your candidate models. Then “internally validate” your models using the holdout sample. More sophisticated methods like cross…