AI wrote the model notebook. Did test data enter preprocessing before the split?

Our view: inspect which data learned each preparation step before trusting a score. A later train/test split cannot undo earlier use of the test rows.

The scikit-learn guide warns that test information used in building a model can make evaluation overly optimistic. It recommends splitting first and learning preprocessing from training data, then applying the learned transformation to both subsets.

Even a mean can carry held-out information

In our invented arithmetic, training values are 0 and 2; the held-out value is 100. The training mean is 1, while the combined mean is 34. Subtracting the combined mean changes the two training values to −34 and −32 instead of −1 and 1. We calculated this difference, without fitting a model or measuring accuracy.

Read where the preparation learned its rule

For an AI-written notebook, our proposed review follows the split, each fit step and the final evaluation. A correct-looking score printout does not show whether a scaler, feature selector or other learned preparation used held-out rows.

Use one learned transformation consistently

Keeping test rows out of fitting does not mean leaving them unprocessed. The guide distinguishes fitting from transforming. Our recommendation is to inspect the data flow rather than treating any preprocessing as contamination.

An arithmetic example is not a score correction

Our numbers show how held-out information can enter a preparation rule. They do not establish the direction or size of a real model’s error. If the workflow is contaminated, reassess it with an appropriate independent evaluation instead of subtracting an invented adjustment from the score.

Published October 14, 2026. Editorial analysis of the current scikit-learn 1.9.1 guide and original mean calculations. We did not run scikit-learn, an AI coding tool, model training, a benchmark or a production dataset.

Sources: scikit-learn · Data leakage.