English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This paper describes the DSBA submissions to the Prompting Large LanguageModels as Explainable Metrics shared task, where systems were submitted to twotracks: small and large summarization tracks. With advanced Large LanguageModels (LLMs) such as GPT-4, evaluating the quality of Natural LanguageGeneration (NLG) has become increasingly paramount. Traditionalsimilarity-based metrics such as BLEU and ROUGE have shown to misalign withhuman evaluation and are ill-suited for open-ended generation tasks. To addressthis issue, we explore the potential capability of LLM-based metrics,especially leveraging open-source LLMs. In this study, wide range of promptsand prompting techniques are systematically analyzed with three approaches:prompting strategy, score aggregation, and explainability. Our research focuseson formulating effective prompt templates, determining the granularity of NLGquality scores and assessing the impact of in-context examples on LLM-basedevaluation. Furthermore, three aggregation strategies are compared to identifythe most reliable method for aggregating NLG quality scores. To examineexplainability, we devise a strategy that generates rationales for the scoresand analyzes the characteristics of the explanation produced by the open-sourceLLMs. Extensive experiments provide insights regarding evaluation capabilitiesof open-source LLMs and suggest effective prompting strategies.

Which is better? Exploring Prompting Strategy For LLM-based Metrics