A Dual-Path Self-Supervised Framework for Polyp Segmentation using Vision Transformers and CNNs

In the field of medical image analysis, accurate polyp segmentation remains a significant challenge due to the limited availability of labeled training data. This paper introduces a novel approach that addresses this issue by leveraging a self-supervised learning framework combined with deep supervision and a dual path architecture of Vision Transformers (ViTs) and Convolutional Neural Networks (CNNs). Our proposed model consists of two paths: Path 1 employs Vision Transformer blocks, with the initial stage utilizing a pre-trained backbone, while subsequent stages further refine these features. Path 2 incorporates self-supervised learning via the Barlow Twins method to capture global context, followed by ResNet blocks and attention-augmented convolution blocks to extract and enhance local features. The outputs from both paths are fused using a cross-attention module, concatenated, and processed through multiple MLP heads. To enhance the segmentation accuracy, deep supervision is applied at multiple decoder stages, utilizing skip connections from the MobileNetV2 backbone. Additionally, we leverage a modified Deep Convolutional Generative Adversarial Network (DCGAN) to generate synthetic polyp images for Barlow Twins pre-training. Experimental results show that our approach outperforms existing methods in polyp segmentation, achieving higher accuracy and robustness in challenging scenarios. Specifically, our model achieves an average Dice score of 94% and an average Intersection over Union (IoU) of 89%, significantly surpassing state-of-the-art methods.