English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

We investigate the usefulness of generative Large Language Models (LLMs) ingenerating training data for cross-encoder re-rankers in a novel direction:generating synthetic documents instead of synthetic queries. We introduce a newdataset, ChatGPT-RetrievalQA, and compare the effectiveness of modelsfine-tuned on LLM-generated and human-generated data. Data generated withgenerative LLMs can be used to augment training data, especially in domainswith smaller amounts of labeled data. We build ChatGPT-RetrievalQA based on anexisting dataset, human ChatGPT Comparison Corpus (HC3), consisting of publicquestion collections with human responses and answers from ChatGPT. Wefine-tune a range of cross-encoder re-rankers on either human-generated orChatGPT-generated data. Our evaluation on MS MARCO DEV, TREC DL'19, and TRECDL'20 demonstrates that cross-encoder re-ranking models trained on ChatGPTresponses are statistically significantly more effective zero-shot re-rankersthan those trained on human responses. In a supervised setting, thehuman-trained re-rankers outperform the LLM-trained re-rankers. Our novelfindings suggest that generative LLMs have high potential in generatingtraining data for neural retrieval models. Further work is needed to determinethe effect of factually wrong information in the generated responses and testour findings' generalizability with open-source LLMs. We release our data,code, and cross-encoders checkpoints for future work.

Generating Synthetic Documents for Cross-Encoder Re-Rankers: A Comparative Study of ChatGPT and Human Experts