Ontology-Driven Bayesian Networks for Dynamic Scene Understanding

This paper describes how an ontology consisting of a ground truth schema and a set of annotated training sequences can be used to train the structure and parameters of Bayesian networks for event recognition. It is shown how the performance of such networks can be improved by augmenting the original ontology with visual object detection, appearance modelling and tracking methods. The integration of these different sources of evidence is optimised with reference to the syntactic and semantic constraints of the ontology. Through the application of these techniques to a visual surveillance problem, it is shown how high-level event, object and scenario properties may be inferred on the basis of the visual content descriptors and an ontology of states, roles, situations and scenarios which is derived from a pre-defined ground truth schema. Performance analysis of the resulting framework allows alternative ontologies to be compared for their self-consistency and realisability in terms of the different visual detection and tracking modules.