English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

During the Italian research assessment exercise, the national agency ANVURperformed an experiment to assess agreement between grades attributed tojournal articles by informed peer review (IR) and by bibliometrics. A sample ofarticles was evaluated by using both methods and agreement was analyzed byweighted Cohen's kappas. ANVUR presented results as indicating an overall'good' or 'more than adequate' agreement. This paper re-examines the experimentresults according to the available statistical guidelines for interpretingkappa values, by showing that the degree of agreement, always in the range0.09-0.42 has to be interpreted, for all research fields, as unacceptable, pooror, in a few cases, as, at most, fair. The only notable exception, confirmedalso by a statistical meta-analysis, was a moderate agreement for economics andstatistics (Area 13) and its sub-fields. We show that the experiment protocoladopted in Area 13 was substantially modified with respect to all the otherresearch fields, to the point that results for economics and statistics have tobe considered as fatally flawed. The evidence of a poor agreement supports theconclusion that IR and bibliometrics do not produce similar results, and thatthe adoption of both methods in the Italian research assessment possiblyintroduced systematic and unknown biases in its final results. The conclusionreached by ANVUR must be reversed: the available evidence does not justify atall the joint use of IR and bibliometrics within the same research assessmentexercise.

Do they agree? Bibliometric evaluation vs informed peer review in the Italian research assessment exercise