English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Task-Oriented Dialogue (TOD) systems are drawing more and more attention inrecent studies. Current methods focus on constructing pre-trained models orfine-tuning strategies while the evaluation of TOD is limited by a policymismatch problem. That is, during evaluation, the user utterances are from theannotated dataset while these utterances should interact with previousresponses which can have many alternatives besides annotated texts. Therefore,in this work, we propose an interactive evaluation framework for TOD. We firstbuild a goal-oriented user simulator based on pre-trained models and then usethe user simulator to interact with the dialogue system to generate dialogues.Besides, we introduce a sentence-level and a session-level score to measure thesentence fluency and session coherence in the interactive evaluation.Experimental results show that RL-based TOD systems trained by our proposeduser simulator can achieve nearly 98% inform and success rates in theinteractive evaluation of MultiWOZ dataset and the proposed scores measure theresponse quality besides the inform and success rates. We are hoping that ourwork will encourage simulator-based interactive evaluations in the TOD task.

Is MultiWOZ a Solved Task? An Interactive TOD Evaluation Framework with User Simulator