Evaluating reasoning systems

A review of the literature on evaluating reasoning systems reveals that it is a very broad area with wide variation in depth and breadth of research on metrics and tests. This is the second of a two-part series that begins to bring order to the area by categorizing reasoning systems according to their capabilities. These can be used as a basis for evaluating and testing reasoning systems claiming to be in each category. Capabilities are analyzed along the dimensions of representation languages and inference. The first part introduces information metrology, model theory, and inference to facilitate understanding of the reasoning categories presented. It also groups representation languages by their relation to first-order logic, and model-theoretic properties, such as soundness and completeness. This part examines inference procedures, dividing them into deduction, induction, abduction, and analogical reasoning. It explains the subcategories and characteristics of each, and concludes with recommendations for future work.