English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Many peer-review venues are either using or looking to use algorithms toassign submissions to reviewers. The crux of such automated approaches is thenotion of the "similarity score"--a numerical estimate of the expertise of areviewer in reviewing a paper--and many algorithms have been proposed tocompute these scores. However, these algorithms have not been subjected to aprincipled comparison, making it difficult for stakeholders to choose thealgorithm in an evidence-based manner. The key challenge in comparing existingalgorithms and developing better algorithms is the lack of the publiclyavailable gold-standard data that would be needed to perform reproducibleresearch. We address this challenge by collecting a novel dataset of similarityscores that we release to the research community. Our dataset consists of 477self-reported expertise scores provided by 58 researchers who evaluated theirexpertise in reviewing papers they have read previously.  We use this data to compare several popular algorithms employed in computerscience conferences and come up with recommendations for stakeholders. Our mainfindings are as follows. First, all algorithms make a non-trivial amount oferror. For the task of ordering two papers in terms of their relevance for areviewer, the error rates range from 12%-30% in easy cases to 36%-43% in hardcases, highlighting the vital need for more research on thesimilarity-computation problem. Second, most existing algorithms are designedto work with titles and abstracts of papers, and in this regime the Specter+MFRalgorithm performs best. Third, to improve performance, it may be important todevelop modern deep-learning based algorithms that can make use of the fulltexts of papers: the classical TD-IDF algorithm enhanced with full texts ofpapers is on par with the deep-learning based Specter+MFR that cannot make useof this information.

A Gold Standard Dataset for the Reviewer Assignment Problem