Copepods Image-Skeleton Pairing with N-Pair Loss

In this study, we propose a novel method to determine whether a copepod image belongs to the same morphological case as a one-channel binary hand-drawn skeleton. This method can be used for indexing and two-modal data matching, allowing hand-drawn skeletons to be aligned with specific functional trait representations. Our problem is particularly challenging due to the small dataset size, morphological case imbalance, and morphological ambiguities. To address these issues, we employ two convolutional neural networks to encode images and skeletons into vector representations. We then refine these encodings using N-Pair Loss, training two fully connected layers to compare the encoded image with skeletons from different morphological cases. This contrastive learning approach leverages additional comparative information, helping the model distinguish subtle morphological variations more effectively. For experiments we used an image dataset of 1969 copepod specimens. The results demonstrated that our method outperforms traditional training approach, effectively addressing the challenges of small, imbalanced, and complex datasets and offering a more robust latent space representation. Moreover, it can be extended to datasets with additional morphological cases and different input types, making it applicable to taxonomic studies, particularly those focusing on functional traits.