Premium
Maximum entropy inference for mixed continuous‐discrete variables
Author(s) -
Singer Hermann
Publication year - 2010
Publication title -
international journal of intelligent systems
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.291
H-Index - 87
eISSN - 1098-111X
pISSN - 0884-8173
DOI - 10.1002/int.20411
Subject(s) - joint probability distribution , principle of maximum entropy , mathematics , marginal distribution , conditional probability distribution , conditional probability , probability distribution , entropy (arrow of time) , inference , multivariate normal distribution , multivariate statistics , computer science , mathematical economics , statistics , artificial intelligence , random variable , physics , quantum mechanics
We represent knowledge by probability distributions of mixed continuous and discrete variables. From the joint distribution of all items, one can compute arbitrary conditional distributions, which may be used for prediction. However, in many cases only some marginal distributions, inverse probabilities, or moments are known. Under these conditions, a principle is needed to determine the full joint distribution of all variables. The principle of maximum entropy (Jaynes, Phys Rev 1957;106:620–630 and 1957;108:171–190; Jaynes, Probability Theory—The Logic of Science , Cambridge, UK: Cambridge University Press, 2003; Haken, Synergetics , Berlin: Springer‐Verlag, 1977; Guiasu and Shenitzer, Math Intell 1985;117:83–106) ensures an unbiased estimation of the full multivariate relationships by using only known facts. For the case of discrete variables, the expert shell SPIRIT implements this approach (cf. Rödder, Artif Intell 2000;117:83–106; Rödder and Meyer, in Proceedings of the 12th Conference on Uncertainty in Artificial Intelligence , San Francisco, CA, 2006; Rödder et al., Logical J IGPL 2006;14(3):483–500). In this paper, the approach is generalized to continuous and mixed continuous‐discrete distributions and applied to the problem of credit scoring. © 2010 Wiley Periodicals, Inc.