English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The recent performance leap of Large Language Models (LLMs) opens up newopportunities across numerous industrial applications and domains. However,erroneous generations, such as false predictions, misinformation, andhallucination made by LLMs, have also raised severe concerns for thetrustworthiness of LLMs', especially in safety-, security- andreliability-sensitive scenarios, potentially hindering real-world adoptions.While uncertainty estimation has shown its potential for interpreting theprediction risks made by general machine learning (ML) models, little is knownabout whether and to what extent it can help explore an LLM's capabilities andcounteract its undesired behavior. To bridge the gap, in this paper, weinitiate an exploratory study on the risk assessment of LLMs from the lens ofuncertainty. In particular, we experiment with twelve uncertainty estimationmethods and four LLMs on four prominent natural language processing (NLP) tasksto investigate to what extent uncertainty estimation techniques could helpcharacterize the prediction risks of LLMs. Our findings validate theeffectiveness of uncertainty estimation for revealing LLMs'uncertain/non-factual predictions. In addition to general NLP tasks, weextensively conduct experiments with four LLMs for code generation on twodatasets. We find that uncertainty estimation can potentially uncover buggyprograms generated by LLMs. Insights from our study shed light on future designand development for reliable LLMs, facilitating further research towardenhancing the trustworthiness of LLMs.

Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Models