Multitask Analysis Method for Tongue Image Based on Edge Computing

In response to the application scenarios of modernized Traditional Chinese Medicine (TCM) diagnostic and treatment equipment moving towards the user end, an effort has been made to enhance the user-friendliness of TCM diagnostic and treatment devices. This involves introducing the concept of edge computing into the mobile tongue diagnosis instrument, shifting the tasks of tongue image acquisition and analysis to portable auxiliary diagnostic devices. To improve the efficiency of edge computing devices in handling tongue image analysis tasks, a multi-task network model based on a lightweight network backbone is proposed. The model utilizes the lightweight feature extraction backbone of MobileNet to provide feature encoding for both the semantic segmentation branch and the multi-label classification branch. The semantic segmentation branch adopts a skip-layer connection structure with multi-scale feature maps, and an attention mechanism is incorporated into the classification branch to fuse the feature maps from the segmentation branch. This achieves tongue segmentation and multi-label classification tasks in a computationally efficient environment. The model achieves a pixel accuracy of 85.3% in semantic segmentation and an accuracy of 95.6% in multi-label classification. The network’s forward propagation speed on edge computing platforms reaches 7 frames per second (FPS). The proposed lightweight network backbone multi-task network model ensures a significant improvement in processing efficiency while maintaining the accuracy of segmentation and classification tasks. Additionally, the model exhibits advantages in terms of quantity and scale, saving both storage and computational resources. It not only enhances the accuracy and efficiency of tongue image real-time analysis in edge computing scenarios but also reduces the processing time, providing excellent precision and inference speed.