What is Out-of-Sample?

Out-of-Sample

Data that was held back and not used when the rules were built. The results on that segment serve as the real test. If that segment is examined repeatedly while the rules are refined, it is no longer held-back data, so the principle is to check it only once, at the end.