English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this paper, we propose an Omni-perception Pre-Trainer (OPT) forcross-modal understanding and generation, by jointly modeling visual, text andaudio resources. OPT is constructed in an encoder-decoder framework, includingthree single-modal encoders to generate token-based embeddings for eachmodality, a cross-modal encoder to encode the correlations among the threemodalities, and two cross-modal decoders to generate text and imagerespectively. For the OPT's pre-training, we design a multi-task pretextlearning scheme to model multi-modal resources from three different datagranularities, \ie, token-, modality-, and sample-level modeling, through whichOPT learns to align and translate among different modalities. The pre-trainingtask is carried out on a large amount of image-text-audio triplets from OpenImages. Experimental results show that OPT can learn strong image-text-audiomulti-modal representations and achieve promising results on a variety ofcross-modal understanding and generation tasks.

OPT: Omni-Perception Pre-Trainer for Cross-Modal Understanding and Generation