English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large multi-modal models (LMMs) have demonstrated promising intelligenceowing to the rapid development of pre-training techniques. However, theirfine-grained cross-modal alignment ability is constrained by the coarsealignment in image-text pairs. This limitation hinders awareness offine-grained concepts, resulting in sub-optimal performance. In this paper, wepropose a multi-modal conceptual knowledge base, named M2ConceptBase, whichaims to provide fine-grained alignment between images and concepts.Specifically, M2ConceptBase models concepts as nodes, associating each withrelevant images and detailed text, thereby enhancing LMMs' cross-modalalignment with rich conceptual knowledge. To collect concept-image andconcept-description alignments, we propose a context-aware multi-modal symbolgrounding approach that considers context information in existing large-scaleimage-text pairs with respect to each concept. A cutting-edge large languagemodel supplements descriptions for concepts not grounded via our symbolgrounding approach. Finally, our M2ConceptBase contains more than 951K imagesand 152K concepts, each associating with an average of 6.27 images and a singledetailed description. We conduct experiments on the OK-VQA task, demonstratingthat our M2ConceptBase facilitates the model in achieving state-of-the-artperformance. Moreover, we construct a comprehensive benchmark to evaluate theconcept understanding of LMMs and show that M2ConceptBase could effectivelyimprove LMMs' concept understanding and cross-modal alignment abilities.

M2ConceptBase: A Fine-grained Aligned Multi-modal Conceptual Knowledge Base