Risk Assessment for Model and Simulation Credibility Characteristics

The Department of Defense (DoD) uses the principles of risk assessment extensively throughout the acquisition process to identify, manage, and mitigate risk in many areas including software (both simulation software and software embedded in materiel systems). The approach to assessing the risks associated with using results from Models and Simulations (M&S) to influence decision-making that is described here is used by the Naval Air Systems Command (NAVAIR) 5.4H Verification, Validation, and Accreditation office. The focus of the risk assessment is on ten specific characteristics of M&S Capability, Accuracy, and Usability that have the most potential impact on the intended uses. In this method, the characteristics of the M&S are evaluated against standard criteria, and then assigned a color-rating of Green, Yellow, or Red based on the actual data obtained from the M&S itself. A simple computation is then used to weigh the ratings of the characteristics into the likelihood of error aspect of the risk of using the M&S. A thorough understanding of the M&S and its application is a prerequisite to using this approach since the analyst is required to provide a technical justification for the rating assigned to a characteristic. Should the assigned rating for a characteristic of the M&S not be green, the analyst is required to recommend actions and/or mitigations for the M&S developer to take that would improve the rating to green.