Culling the Herd: Using Real-World Randomized Experiments to Measure Social Bias with Known Costly Goods

Peer-ratings have become increasingly important sources of product information, particularly in markets for ”information goods.” However, in spite of the increasing prevalence of this information, there are relatively few academic studies that analyze the impact of peer-ratings on consumers transacting in ”real world” marketplaces. In this paper, we partner with a major cable company to analyze the impact of peer-ratings in a real-world Video-on-Demand market where consumer participation is organic and where movies are costly and well-known to consumers. After experimentally manipulating the initial conditions of product information displayed to consumers, we find that, consistent with the prior literature, peer-ratings influence consumer behavior independently from underlying product quality. However, we also find that, in contrast to the prior literature, at least in our setting there is little evidence of long-term bias due to herding effects. Specifically, when movies are artificially promoted or demoted in peer-rating lists, subsequent reviews cause them to return to their true quality position relatively quickly. One explanation for this difference is that consumers in our empirical setting likely had more outside