Open AccessMIDST: an enhanced development environment that improves the maintainability of a data science analysis
Author(s) -
Jeffrey S. Saltz,
Kevin Crowston,
Robert Heckman,
Yatish Hegde
Publication year - 2021
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.411
H-Index - 13
ISSN - 2182-7796
DOI - 10.12821/ijispm080301
Subject(s) - maintainability , field (mathematics) , computer science , modular design , software engineering , data science , code (set theory) , data analysis , systems engineering , engineering , data mining , programming language , mathematics , set (abstract data type) , pure mathematics
With the increasing ability to generate actionable insight from data, the field of data science has seen significant growth. As more teams develop data science solutions, the analytical code they develop will need to be enhanced in the future, by an existing or a new team member. Thus, the importance of being able to easily maintain and enhance the code required for an analysis will increase. However, to date, there has been minimal research on the maintainability of an analysis done by a data science team. To help address this gap, data science maintainability was explored by (1) creating a data science maintainability model, (2) creating a new tool, called MIDST (Modular Interactive Data Science Tool), that aims to improve data science maintainability, and then (3) conducting a mixed method experiment to evaluate MIDST. The new tool aims to improve the ability of a team member to update and rerun an existing data science analysis by providing a visual data flow view of the analysis within an integrated code and computational environment. Via an analysis of the quantitative and qualitative survey results, the experiment found that MIDST does help improve the maintainability of an analysis. Thus, this research demonstrates the importance of enhanced tools tohelp improve the maintainability of data science projects.