English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Prompt learning has demonstrated impressive efficacy in the fine-tuning ofmultimodal large models to a wide range of downstream tasks. Nonetheless,applying existing prompt learning methods for the diagnosis of neurologicaldisorder still suffers from two issues: (i) existing methods typically treatall patches equally, despite the fact that only a small number of patches inneuroimaging are relevant to the disease, and (ii) they ignore the structuralinformation inherent in the brain connection network which is crucial forunderstanding and diagnosing neurological disorders. To tackle these issues, weintroduce a novel prompt learning model by learning graph prompts during thefine-tuning process of multimodal large models for diagnosing neurologicaldisorders. Specifically, we first leverage GPT-4 to obtain relevant diseaseconcepts and compute semantic similarity between these concepts and allpatches. Secondly, we reduce the weight of irrelevant patches according to thesemantic similarity between each patch and disease-related concepts. Moreover,we construct a graph among tokens based on these concepts and employ a graphconvolutional network layer to extract the structural information of the graph,which is used to prompt the pre-trained multimodal large models for diagnosingneurological disorders. Extensive experiments demonstrate that our methodachieves superior performance for neurological disorder diagnosis compared withstate-of-the-art methods and validated by clinicians.

MMGPL: Multimodal Medical Data Analysis with Graph Prompt Learning