English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Learning from Preferential Feedback (LfPF) plays an essential role intraining Large Language Models, as well as certain types of interactivelearning agents. However, a substantial gap exists between the theory andapplication of LfPF algorithms. Current results guaranteeing the existence ofoptimal policies in LfPF problems assume that both the preferences andtransition dynamics are determined by a Markov Decision Process. We introducethe Direct Preference Process, a new framework for analyzing LfPF problems inpartially-observable, non-Markovian environments. Within this framework, weestablish conditions that guarantee the existence of optimal policies byconsidering the ordinal structure of the preferences. Using the vonNeumann-Morgenstern Expected Utility Theorem, we show that the DirectPreference Process generalizes the standard reinforcement learning problem. Ourfindings narrow the gap between the empirical success and theoreticalunderstanding of LfPF algorithms and provide future practitioners with thetools necessary for a more principled design of LfPF agents.

Conditions on Preference Relations that Guarantee the Existence of Optimal Policies