English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Every day, the human brain processes an immense volume of visual information,relying on intricate neural mechanisms to perceive and interpret these stimuli.Recent breakthroughs in functional magnetic resonance imaging (fMRI) haveenabled scientists to extract visual information from human brain activitypatterns. In this study, we present an innovative method for decoding brainactivity into meaningful images and captions, with a specific focus on braincaptioning due to its enhanced flexibility as compared to brain decoding intoimages. Our approach takes advantage of cutting-edge image captioning modelsand incorporates a unique image reconstruction pipeline that utilizes latentdiffusion models and depth estimation. We utilized the Natural Scenes Dataset,a comprehensive fMRI dataset from eight subjects who viewed images from theCOCO dataset. We employed the Generative Image-to-text Transformer (GIT) as ourbackbone for captioning and propose a new image reconstruction pipeline basedon latent diffusion models. The method involves training regularized linearregression models between brain activity and extracted features. Additionally,we incorporated depth maps from the ControlNet model to further guide thereconstruction process. We evaluate our methods using quantitative metrics forboth generated captions and images. Our brain captioning approach outperformsexisting methods, while our image reconstruction pipeline generates plausibleimages with improved spatial relationships. In conclusion, we demonstratesignificant progress in brain decoding, showcasing the enormous potential ofintegrating vision and language to better understand human cognition. Ourapproach provides a flexible platform for future research, with potentialapplications in various fields, including neural art, style transfer, andportable devices.

Brain Captioning: Decoding human brain activity into images and text