English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Scene text removal (STR) aims at replacing text strokes in natural sceneswith visually coherent backgrounds. Recent STR approaches rely on iterativerefinements or explicit text masks, resulting in higher complexity andsensitivity to the accuracy of text localization. Moreover, most existing STRmethods utilize convolutional neural networks (CNNs) for feature representationwhile the potential of vision Transformers (ViTs) remains largely unexplored.In this paper, we propose a simple-yet-effective ViT-based text eraser, dubbedViTEraser. Following a concise encoder-decoder framework, different types ofViTs can be easily integrated into ViTEraser to enhance the long-rangedependencies and global reasoning. Specifically, the encoder hierarchicallymaps the input image into the hidden space through ViT blocks and patchembedding layers, while the decoder gradually upsamples the hidden features tothe text-erased image with ViT blocks and patch splitting layers. As ViTEraserimplicitly integrates text localization and inpainting, we propose a novelend-to-end pretraining method, termed SegMIM, which focuses the encoder anddecoder on the text box segmentation and masked image modeling tasks,respectively. To verify the effectiveness of the proposed methods, wecomprehensively explore the architecture, pretraining, and scalability of theViT-based encoder-decoder for STR, which provides deep insights into theapplication of ViT to STR. Experimental results demonstrate that ViTEraser withSegMIM achieves state-of-the-art performance on STR by a substantial margin.Furthermore, the extended experiment on tampered scene text detectiondemonstrates the generality of ViTEraser to other tasks. We believe this papercan inspire more research on ViT-based STR approaches. Code will be availableat https://github.com/shannanyinxiang/ViTEraser.

ViTEraser: Harnessing the Power of Vision Transformers for Scene Text Removal with SegMIM Pretraining