English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large Language Models (LLMs) have demonstrated great capabilities in solvinga wide range of tasks in a resource-efficient manner through prompting, whichdoes not require task-specific training, but suffers from performancefluctuation when there are multiple prompt candidates. Previous works haveintroduced gradient-free probability-based prompt selection methods that aim tochoose the optimal prompt among the candidates for a given task but fail toprovide a comprehensive and fair comparison between each other. In this paper,we propose a unified framework to interpret and evaluate the existingprobability-based prompt selection methods by performing extensive experimentson 13 common NLP tasks. We find that all existing methods can be unified intosome variant of the method that maximizes the mutual information between theinput and the corresponding model output (denoted as MI). Using the finding, wedevelop several variants of MI and increases the effectiveness of the bestprompt selection method from 87.79% to 94.98%, measured as the ratio of theperformance of the selected prompt to that of the optimal oracle prompt.Furthermore, we propose a novel calibration method called Calibration byMarginalization (CBM) that is orthogonal to existing methods and helps increasethe prompt selection effectiveness of the best method by 99.44%. The code anddatasets used in our work will be released athttps://github.com/soheeyang/unified-prompt-selection.

Improving Probability-based Prompt Selection Through Unified Evaluation and Analysis