English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Image reconstruction and captioning from brain activity evoked by visualstimuli allow researchers to further understand the connection between thehuman brain and the visual perception system. While deep generative models haverecently been employed in this field, reconstructing realistic captions andimages with both low-level details and high semantic fidelity is still achallenging problem. In this work, we propose UniBrain: Unify ImageReconstruction and Captioning All in One Diffusion Model from Human BrainActivity. For the first time, we unify image reconstruction and captioning fromvisual-evoked functional magnetic resonance imaging (fMRI) through a latentdiffusion model termed Versatile Diffusion. Specifically, we transform fMRIvoxels into text and image latent for low-level information and guide thebackward diffusion process through fMRI-based image and text conditions derivedfrom CLIP to generate realistic captions and images. UniBrain outperformscurrent methods both qualitatively and quantitatively in terms of imagereconstruction and reports image captioning results for the first time on theNatural Scenes Dataset (NSD) dataset. Moreover, the ablation experiments andfunctional region-of-interest (ROI) analysis further exhibit the superiority ofUniBrain and provide comprehensive insight for visual-evoked brain decoding.

UniBrain: Unify Image Reconstruction and Captioning All in One Diffusion Model from Human Brain Activity