English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Given the rapid ascent of large language models (LLMs), we study thequestion: (How) can large language models help in reviewing of scientificpapers or proposals? We first conduct some pilot studies where we find that (i)GPT-4 outperforms other LLMs (Bard, Vicuna, Koala, Alpaca, LLaMa, Dolly,OpenAssistant, StableLM), and (ii) prompting with a specific question (e.g., toidentify errors) outperforms prompting to simply write a review. With theseinsights, we study the use of LLMs (specifically, GPT-4) for three tasks:  1. Identifying errors: We construct 13 short computer science papers eachwith a deliberately inserted error, and ask the LLM to check for thecorrectness of these papers. We observe that the LLM finds errors in 7 of them,spanning both mathematical and conceptual errors.  2. Verifying checklists: We task the LLM to verify 16 closed-ended checklistquestions in the respective sections of 15 NeurIPS 2022 papers. We find thatacross 119 {checklist question, paper} pairs, the LLM had an 86.6% accuracy.  3. Choosing the "better" paper: We generate 10 pairs of abstracts,deliberately designing each pair in such a way that one abstract was clearlysuperior than the other. The LLM, however, struggled to discern theserelatively straightforward distinctions accurately, committing errors in itsevaluations for 6 out of the 10 pairs.  Based on these experiments, we think that LLMs have a promising use asreviewing assistants for specific reviewing tasks, but not (yet) for completeevaluations of papers or proposals.

ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewing