English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Due to the lack of paired samples and the low signal-to-noise ratio offunctional MRI (fMRI) signals, reconstructing perceived natural images ordecoding their semantic contents from fMRI data are challenging tasks. In thiswork, we propose, for the first time, a task-agnostic fMRI-based brain decodingmodel, BrainCLIP, which leverages CLIP's cross-modal generalization ability tobridge the modality gap between brain activity, image, and text. Ourexperiments demonstrate that CLIP can act as a pivot for generic brain decodingtasks, including zero-shot visual categories decoding, fMRI-image/textmatching, and fMRI-to-image generation. Specifically, BrainCLIP aims to train amapping network that transforms fMRI patterns into a well-aligned CLIPembedding space by combining visual and textual supervision. Our experimentsshow that this combination can boost the decoding model's performance oncertain tasks like fMRI-text matching and fMRI-to-image generation. On thezero-shot visual category decoding task, BrainCLIP achieves significantlybetter performance than BraVL, a recently proposed multi-modal methodspecifically designed for this task. BrainCLIP can also reconstruct visualstimuli with high semantic fidelity and establishes a new state-of-the-art forfMRI-based natural image reconstruction in terms of high-level semanticfeatures.

BrainCLIP: Bridging Brain and Visual-Linguistic Representation Via CLIP for Generic Natural Visual Stimulus Decoding