Using the Analytic Hierarchy Process for Evaluating Multi-Agent System Architecture Candidates

Although much effort has been spent on suggesting and implementing new architectures of Multi-Agent Systems (MAS), the evaluation and comparison of these has often been done in a rather ad-hoc fashion. We believe that the time has come to start doing this in a more systematic way using established methods. For instance, we argue that it is important to evaluate the architecture candidates for a particular application according to several quality attributes relevant to that application. The architecture that provides the most appropriate balance between these attributes should then be selected. As a case study we investigate the problem of load balancing and overload control of Intelligent Networks and present four MAS architectures that can be used to handle this task. We instantiate each of these and define metrics for the selected quality attributes. The instantiations are studied in simulation experiments and measurements of the metrics are recorded. The measurements are then analyzed using the Analytic Hierarchy Process, which is a basic approach to select the most suitable alternative from a number of alternatives evaluated with respect to several criteria. We illustrate how such analyzes can be used for deciding which architecture candidate is the most appropriate in different situations.