Engineering Serendipity: The Role of Cognitive Similarity in Knowledge Sharing and Knowledge Production

We consider how the cognitive similarity between knowledge-sharing partners affects the knowledge-production process, namely knowledge transfer, creation, and diffusion. We theorize that knowledge production is systematically shaped by the field and intellectual similarity between knowledge-sharing partners’ disciplines of study and domain area interests. To estimate relationships, we designed and executed a natural field experiment at a medical symposium, in which exogenous variation was introduced to provide some of the 15,817 scientist pairs with opportunities for serendipitous, face-to-face encounters. Our data include direct observations of interaction patterns collected using sociometric badges, and detailed longitudinal data on the scientists’ publication records for six years following the symposium. We find both cooperative and competitive effects of cognitive similarity on knowledge production. While knowledge sharing increases the transfer of scientific concepts between scientists with some intellectual overlap, it reduces the diffusion of scientific knowledge between scientists from the same field. In contrast, cognitive similarity does not have a direct effect on knowledge creation, but we find that scientists who have initiated early-stage collaborations with one another are more likely to persist and publish together. The findings suggest that some cognitive similarity between knowledge-sharing partners can boost organizational knowledge production, but too much similarity may impede it.