Train/Test Splits & Data Leakage
A model scores 99% on the house table it studied, rows like price=340k, then flops on the next real listing. It recited rows it had seen, and one slice held back from the start keeps the score honest.
- ▸A 99% score on the rows a model trained on proves it can recite them, not that it learned.
- ▸A table splits at random, but a stock chart splits by time, or the model studies its own future.
- ▸A broken model can score perfectly when test rows leak into training, so distrust anything too good to be true.
Your house-price model just came back from training and posted a 99% score. Before you celebrate, ask what it scored 99% on: the same rows it just studied. It saw every one of those prices, so a high mark there means it can recite, nothing more.
The only number that means anything is the one from rows the model never saw during training. Split your rows in two: the Training set it learns from, and a held-back Test set you grade it on at the end. The two scores pull apart, the recited score stays near perfect while the honest one, measured on rows the model is meeting for the first time, drops to what the model can really do.