English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this paper, we explore the potential of the Contrastive Language-ImagePretraining (CLIP) model in scene text recognition (STR), and establish a novelSymmetrical Linguistic Feature Distillation framework (named CLIP-OCR) toleverage both visual and linguistic knowledge in CLIP. Different from previousCLIP-based methods mainly considering feature generalization on visualencoding, we propose a symmetrical distillation strategy (SDS) that furthercaptures the linguistic knowledge in the CLIP text encoder. By cascading theCLIP image encoder with the reversed CLIP text encoder, a symmetrical structureis built with an image-to-text feature flow that covers not only visual butalso linguistic information for distillation.Benefiting from the naturalalignment in CLIP, such guidance flow provides a progressive optimizationobjective from vision to language, which can supervise the STR featureforwarding process layer-by-layer.Besides, a new Linguistic Consistency Loss(LCL) is proposed to enhance the linguistic capability by consideringsecond-order statistics during the optimization. Overall, CLIP-OCR is the firstto design a smooth transition between image and text for the STR task.Extensiveexperiments demonstrate the effectiveness of CLIP-OCR with 93.8% averageaccuracy on six popular STR benchmarks.Code will be available athttps://github.com/wzx99/CLIPOCR.

Symmetrical Linguistic Feature Distillation with CLIP for Scene Text Recognition