A Multi-branch based Capsule Network with Structural Reparameterization

The capsule network (CapsNet) is an advanced network model. However, the performance of CapsNet on complex datasets (such as CIFAR10) is limited. Improving the network architecture serves as one of the crucial approaches to enhancing the performance of capsule networks. Among these improvements, adopting a multi-branch structure is an effective way to achieve this goal. Unlike directly modifying the architecture, we introduce multi-branch into the routing process and propose multi-branch routing. The capsule network constructed using this routing mechanism is referred to as ReCapsNet. In an effort to mitigate the parameter expansion caused by the multi-branch structure, structural reparameterization is introduced into ReCapsNet. This approach aims to curtail the parameters needed for inference and simplify the architecture. In addition, we utilize the pooling operation based on the attention mechanism to suppress non-important capsules and improve model performance. Experiments on four datasets (CIFAR10, CIFAR100, SVHN, and FMNIST) demonstrate that ReCapsNet has excellent classification performance. Moreover, the reparameterized ReCapsNet model can retain performance comparable to that of the original model, while significantly reducing parameters. Finally, the affine robustness of ReCapsNet is verified on the MNIST dataset and the affNIST dataset.