Diagrammatic Representation and Inference

One might have thought that the issue of use of mental images in reasoning has been put to rest after years of debate, but one would be wrong. The journal Behavioral and Brain Sciences will soon publish a paper by Zenon Pylyshyn, which restates his earlier thesis this time as a null hypothesis that cannot yet be rejected that images do not play any role in reasoning, and that information which people often think they get from images when reasoning is really tacit knowledge that they already have. He does not compare the case of use of mental images with the case when the reasoner uses an external diagram for the same problem, so it is not clear if in those cases also he would claim that the diagram does not play a role. In any case, one might think that this is an issue for psychology, but not for artificial intelligence. However, conceptually, unless one has a robot with a vision system and the robot draws a diagram on a surface and uses its visual system to extract information as it reasons about a situation, any claim that a computer program performs diagrammatic reasoning in some situation has a status similar to that of the claims of human use of mental imagery during reasoning. In parallel with skeptics such as Pylyshyn of the role of images in reasoning, there are those in artificial intelligence who claim that the idea of diagrammatic reasoning by a computer is incoherent: what it means for the internal representation to be an image or a depiction is not clear, and the objections to "seeing" or perceiving information in such an image apply equally to the human and the computer cases. I do not think that the idea of computers performing diagrammatic reasoning is incoherent. I will argue my position on this by characterizing diagrammatic reasoning functionally. Then, I will argue that a computer program can be said to be performing this kind of reasoning if there exists a coherent and non-vacuous level of description of the operations of the program that satisfies the functional properties. Along the way, I will identify diagrammatic reasoning as an instance of a larger class of rea1 Prepared through participation in the Advanced Decision Architectures Collaborative Technology Alliance sponsored by the U.S. Army Research Laboratory under Cooperative Agreement DAAD19-01-2-0009.