English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large Language Models (LLMs), such as ChatGPT, have achieved impressivemilestones in natural language processing (NLP). Despite their impressiveperformance, the models are known to pose important risks. As these models aredeployed in real-world applications, a systematic understanding of differentrisks posed by these models on tasks such as natural language inference (NLI),is much needed. In this paper, we define and formalize two distinct types ofrisk: decision risk and composite risk. We also propose a risk-centricevaluation framework, and four novel metrics, for assessing LLMs on these risksin both in-domain and out-of-domain settings. Finally, we propose arisk-adjusted calibration method called DwD for helping LLMs minimize theserisks in an overall NLI architecture. Detailed experiments, using four NLIbenchmarks, three baselines and two LLMs, including ChatGPT, show both thepractical utility of the evaluation framework, and the efficacy of DwD inreducing decision and composite risk. For instance, when using DwD, anunderlying LLM is able to address an extra 20.1% of low-risk inference tasks(but which the LLM erroneously deems high-risk without risk adjustment) andskip a further 19.8% of high-risk tasks, which would have been answeredincorrectly.

A Formalism and Approach for Improving Robustness of Large Language Models Using Risk-Adjusted Confidence Scores