English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Conversational generative AI has demonstrated remarkable promise forempowering biomedical practitioners, but current investigations focus onunimodal text. Multimodal conversational AI has seen rapid progress byleveraging billions of image-text pairs from the public web, but suchgeneral-domain vision-language models still lack sophistication inunderstanding and conversing about biomedical images. In this paper, we proposea cost-efficient approach for training a vision-language conversationalassistant that can answer open-ended research questions of biomedical images.The key idea is to leverage a large-scale, broad-coverage biomedicalfigure-caption dataset extracted from PubMed Central, use GPT-4 toself-instruct open-ended instruction-following data from the captions, and thenfine-tune a large general-domain vision-language model using a novel curriculumlearning method. Specifically, the model first learns to align biomedicalvocabulary using the figure-caption pairs as is, then learns to masteropen-ended conversational semantics using GPT-4 generated instruction-followingdata, broadly mimicking how a layperson gradually acquires biomedicalknowledge. This enables us to train a Large Language and Vision Assistant forBioMedicine (LLaVA-Med) in less than 15 hours (with eight A100s). LLaVA-Medexhibits excellent multimodal conversational capability and can followopen-ended instruction to assist with inquiries about a biomedical image. Onthree standard biomedical visual question answering datasets, LLaVA-Medoutperforms previous supervised state-of-the-art on certain metrics. Tofacilitate biomedical multimodal research, we will release ourinstruction-following data and the LLaVA-Med model.

LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day