Explanation 1
The live starter code is self-contained and uses only runner-compatible functions.
Loading
Lesson 22 of 22
Assemble provenance, validation, exploration, benchmarks, ARIMA diagnostics, holdout scoring, and responsible communication.
Motivation
An end-to-end case exposes how definitions affect transformations, transformations affect models, and evaluation design affects conclusions.
Why this matters
Assemble provenance, validation, exploration, benchmarks, ARIMA diagnostics, holdout scoring, and responsible communication.
Packages and data
Base stats; dplyr, ggplot2, forecast, and fable are maintainable extensions.
Explanation 1
The live starter code is self-contained and uses only runner-compatible functions.
Explanation 2
Book-only outputs are labelled when the separate source dataset or advanced package was not supplied.
Explanation 3
R code, paths, function names, and formulas remain left-to-right in every locale.
Terminology
Latest time whose information is available when producing a forecast.
Training endpoint
Transparent reference forecast a complex model must earn improvement over.
Last observation or recent mean
Historical split using latest-revised data rather than vintage data available then.
Must not be called genuine real time
Notation and formulas
RMSE = sqrt(mean((actual-forecast)^2)); MAE = mean(abs(actual-forecast))
RMSE emphasizes large errors; MAE weights absolute errors linearly, so model rank can depend on decision loss.
Worked example
Scenario
Which of three candidates forecasts an 18-month Canadian inflation holdout best in the book's frozen dataset?
R check
actual <- c(3.4,3.0,2.8,2.6,2.4,2.2)
naive <- rep(3.5,6)
mean36 <- rep(3.0,6)
arima_fc <- c(3.2,2.9,2.7,2.5,2.4,2.3)
score <- function(f) c(RMSE=sqrt(mean((actual-f)^2)),MAE=mean(abs(actual-f)))
print(rbind(Naive=score(naive),Mean36=score(mean36),ARIMA=score(arima_fc)))That result supports ARIMA for one frozen educational exercise, not universally. Limitation: The study lacks real-time vintages, survey expectations, commodity prices, output gaps, release calendars, parameter-uncertainty integration, and repeated final test origins.
Visual
ARIMA has the lowest reported RMSE and MAE in the single 18-month book exercise.
R connection
Run the self-contained starter code in the protected STATLAB R runner. The code prints an auditable result and avoids network or unrestricted file access.
Live R Lab
Run a self-contained example, verify its output, and explain one limitation for case study: forecasting canadian inflation.
Ready to run
Common mistake
Do not choose the winner on a holdout and report the same score as untouched final evidence.
STATLAB Tip
Write the limitations paragraph before the executive conclusion.
Guided practice
Scenario
Move the forecast origin back six months and record whether the winner changes.
Compare your result with the definition, units, and model assumptions—not only with a target number.
Exercises
Complete these without looking at the selected solutions. More than one defensible program may exist.
Explanation 1
Add a seasonal-naive benchmark.
Explanation 2
Design a rolling-origin results table.
Explanation 3
Write a 150-word memo with three limitations.
Selected solutions
These are compact solution routes. Confirm dimensions, units, and any changed modelling choices.
Explanation 1
Match the seasonal lag to the target definition and verify indexing.
Explanation 2
Include origin, horizon, model, actual, forecast, error, and data vintage.
Explanation 3
State target, horizon, best model, error, uncertainty, and limitations without causal overreach.
Chapter summary
Assemble provenance, validation, exploration, benchmarks, ARIMA diagnostics, holdout scoring, and responsible communication.
Explanation 1
Fix target, horizon, origin, information set, and loss.
Explanation 2
Validate and document data before fitting.
Explanation 3
Compare identical holdouts and benchmarks.
Explanation 4
Communicate scope, uncertainty, and reproducibility.
Terminology
validation
Use in Case Study: Forecasting Canadian Inflation.
training split
Use in Case Study: Forecasting Canadian Inflation.
holdout
Use in Case Study: Forecasting Canadian Inflation.
forecast
Use in Case Study: Forecasting Canadian Inflation.
References and provenance
Safavi (2026), Chapter 22; Statistics Canada (2026); Bank of Canada (2026). Student notes: Mohammad Safavi, Ph.D., STATLAB Academy, Version 1.0.
Resource
The authoritative 125-page English PDF accompanies this native lesson.
DownloadReflection
Name the assumption, evidence you would seek, and how the recommendation might change.
Exit check
Reach 70% to complete the chapter. Explanations appear after submission.
Checkpoint
Question 1 of 2. Answered 0/2. Passing score: 70%.