AI Slop Is Already in Your Training Dataset. I Tested Three Ways to Spot It.

An experiment tests methods for detecting AI-generated content within a training dataset of movie reviews. Three techniques—perplexity, near duplicate similarity, and embedding density—were used to identify synthetic reviews mixed with genuine ones from a 2019 collection. While the detectors successfully flagged most of the AI content, they also misidentified a high number of human-written reviews as being AI-generated. Ultimately, filtering the dataset based on these detection methods resulted in a decrease in the accuracy of a sentiment classification model trained on the cleaned data.