Retrieving and mining professional experience of software practice from grey literature: an exploratory review

Retrieving and mining practitioners’ self‐reports of their professional experience of software practice could provide valuable evidence for research. The authors are, however, unaware of any existing reviews of research conducted in this area. The authors reviewed and classified previous research, and identified insights into the challenges research confronts when retrieving and mining practitioners’ self‐reports of their experience of software practice. They conducted an exploratory review to identify and classify 42 studies. They analysed a selection of those studies for insights on challenges to mining professional experience. They identified only one directly relevant study. Even then this study concerns the software professional's emotional experiences rather than the professional's reporting of behaviour and events occurring during software practice. They discussed the challenges concerning: the prevalence of professional experience; definitions, models and theories; the sparseness of data; units of discourse analysis; annotator agreement; evaluation of the performance of algorithms; and the lack of replications. No directly relevant prior research appears to have been conducted in this area. They discussed the value of reporting negative results in secondary studies. There are a range of research opportunities but also considerable challenges. They formulated a set of guiding questions for further research in this area.