Haptic augmented skin surface generation toward telepalpation from a mobile skin image

Background/purpose Very little is known about the methods of integrating palpation techniques to existing mobile teleskin imaging that delivers low quality tactile information (roughness) for telepalpation. However, no study has been reported yet regarding telehaptic palpation using mobile phone images for teledermatology or teleconsultations of skincare. Methods This study is therefore aimed at introducing a new algorithm accurately reconstructing a haptic augmented skin surface for telehaptic palpation using a low‐cost clip‐on microscope simply attached to a mobile phone. Multiple algorithms such as gradient‐based image enhancement, roughness‐adaptive tactile mask generation, roughness‐enhanced 3D tactile map building, and visual and haptic rendering with a three‐degrees‐of‐freedom (DOF) haptic device were developed and integrated as one system. Results Evaluation experiments have been conducted to test the performance of 3D roughness reconstruction with/without the tactile mask. The results confirm that reconstructed haptic roughness with the tactile mask is superior to the reconstructed haptic roughness without the tactile mask. Additional experiments demonstrate that the proposed algorithm is robust against varying lighting conditions and blurring. In last, a user study has been designed to see the effect of the haptic modality to the existing visual only interface and the results attest that the haptic skin palpation can significantly improve the skin exam performance. Conclusion Mobile image‐based telehaptic palpation technology was proposed, and an initial version was developed. The developed technology was tested with several skin images and the experimental results showed the superiority of the proposed scheme in terms of the performance of haptic augmentation of real skin images.