MST: Adaptive Multi-Scale Tokens Guided Interactive Segmentation

In the field of Industrial Informatics, interactive segmentation has gainedsignificant attention for its application in human-computer interaction anddata annotation. Existing algorithms, however, face challenges in balancing thesegmentation accuracy between large and small targets, often leading to anincreased number of user interactions. To tackle this, a novel multi-scaletoken adaptation algorithm, leveraging token similarity, has been devised toenhance segmentation across varying target sizes. This algorithm utilizes adifferentiable top-k tokens selection mechanism, allowing for fewer tokens tobe used while maintaining efficient multi-scale token interaction. Furthermore,a contrastive loss is introduced to better discriminate between target andbackground tokens, improving the correctness and robustness of the tokenssimilar to the target. Extensive benchmarking shows that the algorithm achievesstate-of-the-art (SOTA) performance compared to current methods. An interactivedemo and all reproducible codes will be released athttps://github.com/hahamyt/mst.