Text2MDT: Extracting Medical Decision Trees from Medical Texts

Knowledge of the medical decision process, which can be modeled as medicaldecision trees (MDTs), is critical to build clinical decision support systems.However, the current MDT construction methods rely heavily on time-consumingand laborious manual annotation. In this work, we propose a novel task,Text2MDT, to explore the automatic extraction of MDTs from medical texts suchas medical guidelines and textbooks. We normalize the form of the MDT andcreate an annotated Text-to-MDT dataset in Chinese with the participation ofmedical experts. We investigate two different methods for the Text2MDT tasks:(a) an end-to-end framework which only relies on a GPT style large languagemodels (LLM) instruction tuning to generate all the node information and treestructures. (b) The pipeline framework which decomposes the Text2MDT task tothree subtasks. Experiments on our Text2MDT dataset demonstrate that: (a) theend-to-end method basd on LLMs (7B parameters or larger) show promisingresults, and successfully outperform the pipeline methods. (b) Thechain-of-thought (COT) prompting method \cite{Wei2022ChainOT} can improve theperformance of the fine-tuned LLMs on the Text2MDT test set. (c) thelightweight pipelined method based on encoder-based pretrained models canperform comparably with LLMs with model complexity two magnititudes smaller.Our Text2MDT dataset is open-sourced at\url{https://tianchi.aliyun.com/dataset/95414}, and the source codes areopen-sourced at \url{https://github.com/michael-wzhu/text2dt}.