The problem
Can a CNN reliably tell apart conforming and defective cast metal pump impellers — a visual inspection task normally done by a human on a production line? 7,348 grayscale images, already split into training and test sets, were used to find out.
Key results
Training the CNN
A lightweight CNN — three convolutional blocks (16→32→64 channels) with pooling and dropout — was trained for 10 epochs on 6,633 training images, downscaled to 128x128. Training accuracy climbed steadily from 85% to 99.5%, with a healthy, consistently decreasing loss curve.
A perfect score — and why that's a red flag
Evaluated on the 715-image test set, the model scored 1.0000 across accuracy, precision, recall, and F1. Every single test image was classified correctly.
Auditing for data leakage
Exact duplicates. Hashing every image in both splits and comparing them directly found 64 images that are byte-for-byte identical between train and test — about 9% of the entire test set, already seen by the model before being "evaluated" on the same content.
Near-duplicates. 64 exact duplicates alone don't explain a perfect score across all 715 test images. Comparing the average pixel-by-pixel difference between test images and their closest training match showed several distances in the 5–25 range — well below the baseline established by comparing random, genuinely unrelated training image pairs against each other (5th percentile ≈ 28, mean ≈ 42). Several test images were more similar to a training image than the most similar 5% of entirely unrelated pairs — strong evidence of near-identical photos, likely consecutive shots of the same physical part, split across train and test.
The real finding
The perfect score is inflated by leakage baked into how the public dataset was split — not by an error in this project's training or evaluation code. No re-split was performed to manufacture a more conservative number; the leakage was investigated, quantified, and reported transparently instead.