English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Generative models have been widely applied to solve extractive tasks, whereparts of the input is extracted to form the desired output, and achievedsignificant success. For example, in extractive question answering (QA),generative models have constantly yielded state-of-the-art results. In thiswork, we identify the issue of tokenization inconsistency that is commonlyneglected in training these models. This issue damages the extractive nature ofthese tasks after the input and output are tokenized inconsistently by thetokenizer, and thus leads to performance drop as well as hallucination. Wepropose a simple yet effective fix to this issue and conduct a case study onextractive QA. We show that, with consistent tokenization, the model performsbetter in both in-domain and out-of-domain datasets, with a notable average of+1.7 F2 gain when a BART model is trained on SQuAD and evaluated on 8 QAdatasets. Further, the model converges faster, and becomes less likely togenerate out-of-context answers. With these findings, we would like to call formore attention on how tokenization should be done when solving extractive tasksand recommend applying consistent tokenization during training.

Tokenization Consistency Matters for Generative Models on Extractive NLP Tasks