English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The key challenge in cross-modal retrieval is to find similarities betweenobjects represented with different modalities, such as image and text. However,each modality embeddings stem from non-related feature spaces, which causes thenotorious 'heterogeneity gap'. Currently, many cross-modal systems try tobridge the gap with self-attention. However, self-attention has been widelycriticized for its quadratic complexity, which prevents many real-lifeapplications. In response to this, we propose T-EMDE - a neural densityestimator inspired by the recently introduced Efficient Manifold DensityEstimator (EMDE) from the area of recommender systems. EMDE operates onsketches - representations especially suitable for multimodal operations.However, EMDE is non-differentiable and ingests precomputed, static embeddings.With T-EMDE we introduce a trainable version of EMDE which allows fullend-to-end training. In contrast to self-attention, the complexity of oursolution is linear to the number of tokens/segments. As such, T-EMDE is adrop-in replacement for the self-attention module, with beneficial influence onboth speed and metric performance in cross-modal settings. It facilitatescommunication between modalities, as each global text/image representation isexpressed with a standardized sketch histogram which represents the samemanifold structures irrespective of the underlying modality. We evaluate T-EMDEby introducing it into two recent cross-modal SOTA models and achieving newstate-of-the-art results on multiple datasets and decreasing model latency byup to 20%.

T-EMDE: Sketching-based global similarity for cross-modal retrieval