EP.TU.157Can Twitter Attention Predict Citation Metrics? A Machine Learning Aided Analysis

Aims Surgical journals have developed social media profiles to increase engagement though it remains unclear as to whether social media attention indices for publications act as a surrogate or predictor of traditional citation metrics. This study used machine learning to determine if there is a relationship between Twitter mentions and number of citations for surgical publications. Methods We identified all original research and review papers published in Annals of Surgery, BJS and JAMA Surgery in 2019. Citations data and Twitter mentions were retrieved and the Spearman rank coefficient was used to determine degree of correlation between the two variables. An unsupervised machine-learning hierarchical clustering algorithm was used to define clusters of outlying papers. Quantitative and qualitative analysis of the clusters was completed. Results 413 papers were selected. Median number of citations was 7 (IQR 3-14), median number of Twitter mentions was 40 (IQR 15-79). No correlation between Twitter mentions and number of citations was observed (Spearman’s rho 0.076 p-value 0.124). Cluster analysis identified one large (cluster 2, 367/413 papers) and six small clusters. Analysis of cluster 2 revealed a weak but significant correlation between citations and Twitter mentions (Spearman’s rho 0.107 p-value 0.041). The remaining six clusters were characterised by an out of proportion number of Twitter mentions compared to citations or vice versa. Conclusions Twitter mentions should not be used as a surrogate or predictor of traditional citation metrics. In our database the relationship between social media attention and citations was skewed by a small number of outlying papers.