English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

This paper appraises the concordance between bibliometrics and peer review,by drawing evidence from the data of two experiments realized by the Italiangovernmental agency for research evaluation. The experiments were performed forvalidating the dual system of evaluation, consisting in the interchangeable useof bibliometyrics and peer review, adopted by the agency in the researchassessment exercises. The two experiments were based on stratified randomsamples of journal articles. Each article was scored by bibliometrics and bypeer review. The degree of concordance between the two evaluations is thencomputed. The correct setting of the experiments is defined by developing thedesign-based estimation of the Cohen's kappa coefficient and some testingprocedures for assessing the homogeneity of missing proportions between strata.The results of both experiments show that for each research areas of hardsciences, engineering and life sciences, the degree of agreement betweenbibliometrics and peer review is -- at most -- weak at an individual articlelevel. Thus, the outcome of the experiments does not validate the use of thedual system of evaluation in the Italian research assessments. More in general,the very weak concordance indicates that metrics should not replace peer reviewat the level of individual article. Hence, the use of the dual system ofevaluation for reducing costs might introduce unknown biases in a researchassessment exercise.

On the agreement between bibliometrics and peer review: evidence from the Italian research assessment exercises