English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Despite growing interest in using large language models (LLMs) in healthcare,current explorations do not assess the real-world utility and safety of LLMs inclinical settings. Our objective was to determine whether two LLMs can serveinformation needs submitted by physicians as questions to an informaticsconsultation service in a safe and concordant manner. Sixty six questions froman informatics consult service were submitted to GPT-3.5 and GPT-4 via simpleprompts. 12 physicians assessed the LLM responses' possibility of patient harmand concordance with existing reports from an informatics consultation service.Physician assessments were summarized based on majority vote. For no questionsdid a majority of physicians deem either LLM response as harmful. For GPT-3.5,responses to 8 questions were concordant with the informatics consult report,20 discordant, and 9 were unable to be assessed. There were 29 responses withno majority on "Agree", "Disagree", and "Unable to assess". For GPT-4,responses to 13 questions were concordant, 15 discordant, and 3 were unable tobe assessed. There were 35 responses with no majority. Responses from both LLMswere largely devoid of overt harm, but less than 20% of the responses agreedwith an answer from an informatics consultation service, responses containedhallucinated references, and physicians were divided on what constitutes harm.These results suggest that while general purpose LLMs are able to provide safeand credible responses, they often do not meet the specific information need ofa given question. A definitive evaluation of the usefulness of LLMs inhealthcare settings will likely require additional research on promptengineering, calibration, and custom-tailoring of general purpose models.

Evaluation of GPT-3.5 and GPT-4 for supporting real-world information needs in healthcare delivery