The Model Was Right. The Corrections Still Failed.

A LightGBM model was deployed across production lines to predict and correct overruns mid-job, reducing average overproduction from +24 cases to +2 on historical data. In pilot, the corrections worked: overruns vanished. But a closer look at the production data showed something different. When the order target was reduced by 21 cases, the line overproduced the new target by 18. The system was not being corrected – it was negotiating with the correction, regenerating overrun relative to whatever target the machine was now working toward. Compounding this: corrected jobs left no trace in the training data that a human had intervened, meaning the model, on retraining, would learn to stop flagging the problem it had been built to solve. This talk walks through the pilot data, explains why conservatism in the loss function wasn’t enough, and argues that the real gap isn’t in the model – it’s in the experiment infrastructure that doesn’t exist between a prediction and a PLC. Attendees will leave with a concrete mental model of reflexivity and contamination in industrial ML deployment, a framework for when not to close the automation loop, and specific reasons the standard MLOps playbook (holdouts, impression logging, A/B frameworks) doesn’t transfer to systems that act on physical state.