Open AccessGeometry of the probability simplex and its connection to the maximum entropy method
Author(s) -
Henryk Gzyl,
Frank Nielsen
Publication year - 2020
Publication title -
journal of applied mathematics, statistics and informatics
Language(s) - English
Resource type - Journals
eISSN - 1339-0015
pISSN - 1336-9180
DOI - 10.2478/jamsi-2020-0003
Subject(s) - mathematics , geodesic , exponential family , simplex , finite set , connection (principal bundle) , submanifold , entropy (arrow of time) , exponential function , principle of maximum entropy , geometry , mathematical analysis , physics , statistics , quantum mechanics
The use of geometrical methods in statistics has a long and rich history highlighting many different aspects. These methods are usually based on a Riemannian structure defined on the space of parameters that characterize a family of probabilities. In this paper, we consider the finite dimensional case but the basic ideas can be extended similarly to the infinite-dimensional case. Our aim is to understand exponential families of probabilities on a finite set from an intrinsic geometrical point of view and not through the parameters that characterize some given family of probabilities. For that purpose, we consider a Riemannian geometry defined on the set of positive vectors in a finite-dimensional space. In this space, the probabilities on a finite set comprise a submanifold in which exponential families correspond to geodesic surfaces. We shall also obtain a geometric/dynamic interpretation of Jaynes’ method of maximum entropy.