English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

This paper presents a three-tier modality alignment approach to learningtext-image joint embedding, coined as JEMA, for cross-modal retrieval ofcooking recipes and food images. The first tier improves recipe text embeddingby optimizing the LSTM networks with term extraction and ranking enhancedsequence patterns, and optimizes the image embedding by combining theResNeXt-101 image encoder with the category embedding using wideResNet-50 withword2vec. The second tier modality alignment optimizes the textual-visual jointembedding loss function using a double batch-hard triplet loss with soft-marginoptimization. The third modality alignment incorporates two types ofcross-modality alignments as the auxiliary loss regularizations to furtherreduce the alignment errors in the joint learning of the two modality-specificembedding functions. The category-based cross-modal alignment aims to align theimage category with the recipe category as a loss regularization to the jointembedding. The cross-modal discriminator-based alignment aims to add thevisual-textual embedding distribution alignment to further regularize the jointembedding loss. Extensive experiments with the one-million recipes benchmarkdataset Recipe1M demonstrate that the proposed JEMA approach outperforms thestate-of-the-art cross-modal embedding methods for both image-to-recipe andrecipe-to-image retrievals.

Learning Joint Embedding with Modality Alignments for Cross-Modal Retrieval of Recipes and Food Images