English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Many models have been proposed for vision and language tasks, especially theimage-text retrieval task. All state-of-the-art (SOTA) models in this challengecontained hundreds of millions of parameters. They also were pretrained on alarge external dataset that has been proven to make a big improvement inoverall performance. It is not easy to propose a new model with a novelarchitecture and intensively train it on a massive dataset with many GPUs tosurpass many SOTA models, which are already available to use on the Internet.In this paper, we proposed a compact graph-based framework, named HADA, whichcan combine pretrained models to produce a better result, rather than buildingfrom scratch. First, we created a graph structure in which the nodes were thefeatures extracted from the pretrained models and the edges connecting them.The graph structure was employed to capture and fuse the information from everypretrained model with each other. Then a graph neural network was applied toupdate the connection between the nodes to get the representative embeddingvector for an image and text. Finally, we used the cosine similarity to matchimages with their relevant texts and vice versa to ensure a low inference time.Our experiments showed that, although HADA contained a tiny number of trainableparameters, it could increase baseline performance by more than 3.6% in termsof evaluation metrics in the Flickr30k dataset. Additionally, the proposedmodel did not train on any external dataset and did not require many GPUs butonly 1 to train due to its small number of parameters. The source code isavailable at https://github.com/m2man/HADA.

HADA: A Graph-based Amalgamation Framework in Image-text Retrieval