English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We present MindEye, a novel fMRI-to-image approach to retrieve andreconstruct viewed images from brain activity. Our model comprises two parallelsubmodules that are specialized for retrieval (using contrastive learning) andreconstruction (using a diffusion prior). MindEye can map fMRI brain activityto any high dimensional multimodal latent space, like CLIP image space,enabling image reconstruction using generative models that accept embeddingsfrom this latent space. We comprehensively compare our approach with otherexisting methods, using both qualitative side-by-side comparisons andquantitative evaluations, and show that MindEye achieves state-of-the-artperformance in both reconstruction and retrieval tasks. In particular, MindEyecan retrieve the exact original image even among highly similar candidatesindicating that its brain embeddings retain fine-grained image-specificinformation. This allows us to accurately retrieve images even from large-scaledatabases like LAION-5B. We demonstrate through ablations that MindEye'sperformance improvements over previous methods result from specializedsubmodules for retrieval and reconstruction, improved training techniques, andtraining models with orders of magnitude more parameters. Furthermore, we showthat MindEye can better preserve low-level image features in thereconstructions by using img2img, with outputs from a separate autoencoder. Allcode is available on GitHub.

Reconstructing the Mind's Eye: fMRI-to-Image with Contrastive Learning and Diffusion Priors