Time-Evolving Fairness and Accuracy in Recommendation Systems: A Stereotype-Based Framework for Addressing Trade-Offs

Recent research has raised critical concerns about how offline evaluations of recommender systems (RS) are conducted, particularly regarding issues such as data leakage, temporal bias, and limited metric scope. When it comes to evaluation, the accuracy of recommendations and ranked lists offers a narrow view of the performance, usability, customer satisfaction, and risks of an RS. Among several other desirable properties of an RS, fairness has rightfully gained a prominent place in the research community. Trade-offs have been documented between conflicting metrics, but no work has addressed these as a dynamic time-varying properties. By proposing a novel experimental framework that models accuracy and fairness as evolving, non-static properties of RS, we aim to provide a new way of examining RS properties and present fresh evidence on trade-offs. The significance of this topic lies in its implications for real-world deployment, where fairness across users and items, especially in multi-stakeholder contexts, changes over time and must be balanced with predictive accuracy. By re-framing fairness using stereotype-based control groups, we enable more nuanced and dynamic assessments. We apply this framework across multiple real-world datasets (e.g., MovieLens+IMDb, Amazon), simulating user and item cold-start scenarios in a walk-forward evaluation setup. The results highlight the complex trade-offs between competing metrics and offer actionable insights for the design of fairer, more robust recommender systems.