Dog skin photo study finds performance falls on a stricter test
Researchers grouped images by source to reduce the chance that near-duplicates appeared in both training and testing.
Note: Research using animal images. Independent clinical validation remains pending.
Researchers trained models to classify dog skin photographs and tested how the results changed when images from the same source were kept together. The team audited image similarity and used source information to separate training and test material. That matters because near-duplicate photos can make a model look more capable than it is when it sees a genuinely unfamiliar case.
The selected model scored 94.62% balanced accuracy during cross-validation. It reached 78.06% on the separate internal test of 226 images. Balanced accuracy gives equal weight to the categories even when the dataset contains more examples of some conditions than others.
The images came from repositories. The authors call for independent testing on clinically annotated images before making claims about performance in real clinics.
Why it matters
An evaluation split that keeps related images together gives readers a better idea of where an image classifier may fail.
