English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Multimodal knowledge graphs (MKGs), which intuitively organize information invarious modalities, can benefit multiple practical downstream tasks, such asrecommendation systems, and visual question answering. However, most MKGs arestill far from complete, which motivates the flourishing of MKG reasoningmodels. Recently, with the development of general artificial architectures, thepretrained transformer models have drawn increasing attention, especially formultimodal scenarios. However, the research of multimodal pretrainedtransformer (MPT) for knowledge graph reasoning (KGR) is still at an earlystage. As the biggest difference between MKG and other multimodal data, therich structural information underlying the MKG still cannot be fully leveragedin existing MPT models. Most of them only utilize the graph structure as aretrieval map for matching images and texts connected with the same entity.This manner hinders their reasoning performances. To this end, we propose thegraph Structure Guided Multimodal Pretrained Transformer for knowledge graphreasoning, termed SGMPT. Specifically, the graph structure encoder is adoptedfor structural feature encoding. Then, a structure-guided fusion module withtwo different strategies, i.e., weighted summation and alignment constraint, isfirst designed to inject the structural information into both the textual andvisual features. To the best of our knowledge, SGMPT is the first MPT model formultimodal KGR, which mines the structural information underlying the knowledgegraph. Extensive experiments on FB15k-237-IMG and WN18-IMG, demonstrate thatour SGMPT outperforms existing state-of-the-art models, and prove theeffectiveness of the designed strategies.

Structure Guided Multi-modal Pre-trained Transformer for Knowledge Graph Reasoning