Predicting failure events from crowd‐derived inputs: schedule slips and missed requirements

Systems engineers and project managers have a wide range of tools and approaches they use to assess risk, but these approaches have not helped to reduce failures as much as hoped. Post‐failure analyses may not be enough to prevent a future failure, even at the same organization. In this paper, we present a risk assessment prototype that goes beyond collecting information about the failures or failure causes themselves and aims to consider the human actions that lead to failure. Our method adds “crowd signals” to capture the human actions and behaviors that we know eventually lead to failures. Crowd signals are data derived from a set of questions that members of a project team answer, and are used as input for our prediction models. We collected data from 18 different engineering student projects at Purdue University and built two types of models: one to predict schedule failures and one for technical requirements failure. In both cases, we found that a failure during the previous week increases the likelihood of the same type of failure the following week. Some human behaviors such as students knowing more about their teammates, understanding all implications of their project actions, and not wasting time discussing trivial matters help reduce the likelihood of failures. In contrast, when students are not learning anything new through their involvement with the projects, postponing or cancelling required meetings or tasks, or having problems resurface due to poor solutions, the likelihood of failures increases. In future work, we intend to use the predictive models to provide feedback to the student teams. The work presented in this paper is part of ongoing research efforts to improve risk assessment approaches with the goal to test our methods with industry partners and evaluate the possibilities to integrate with their existing protocols.