Premium
Harmonium Models for Video Classification
Author(s) -
Yang Jun,
Yan Rong,
Liu Yan,
Xing Eric P.
Publication year - 2008
Publication title -
statistical analysis and data mining: the asa data science journal
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.381
H-Index - 33
eISSN - 1932-1872
pISSN - 1932-1864
DOI - 10.1002/sam.103
Subject(s) - computer science , graphical model , artificial intelligence , probabilistic logic , representation (politics) , histogram , benchmark (surveying) , pattern recognition (psychology) , class (philosophy) , image (mathematics) , geodesy , politics , political science , law , geography
Accurate and efficient video classification demands the fusion of multimodal information and the use of intermediate representations. Combining the two ideas into one framework, we propose a series of probabilistic models for video representation and classification using intermediate semantic representations derived from multimodal features of video. On the basis of a class of bipartite undirected graphical models named harmonium, we propose dual‐wing harmonium (DWH) model that represents video shots as latent semantic topics derived by jointly modeling the transcript keywords and color‐histogram features of the data. Our family‐of‐harmonium (FoH) and hierarchical harmonium (HH) model extends DWH by introducing variables representing category labels of data, which allows data representation and classification to be performed in the same model. Our models are among the few attempts of using undirected graphical models for representing and classifying video data. Experiments on a benchmark video collection show different semantic interpretations of video data under our models, as well as superior classification performance in comparison with several directed models. Copyright © 2008 Wiley Periodicals, Inc., A Wiley Company Statistical Analy Data Mining 1: 000‐000, 2008