Open AccessAn Algorithm for Computing Average Mutual Information using Probability Distribution Smoothing
Author(s) -
Amr Goneid
Publication year - 2019
Publication title -
international journal on recent and innovation trends in computing and communication
Language(s) - English
Resource type - Journals
ISSN - 2321-8169
DOI - 10.17762/ijritcc.v7i9.5356
Subject(s) - smoothing , kernel density estimation , estimator , algorithm , computer science , entropy (arrow of time) , histogram , mutual information , computation , multivariate statistics , kernel smoother , mathematics , statistics , kernel method , artificial intelligence , machine learning , physics , quantum mechanics , radial basis function kernel , support vector machine , image (mathematics) , computer vision
There is continuing interest in using Average Mutual Information (AMI) to quantify the pair-wise distance between dataset profiles. Among several algorithms used to find a numerical estimation of AMI, the histogram method is the most common since it provides simplicity and least cost. However, this algorithm is known to underestimate the computed entropies and to overestimate the resulting AMI. Kernel Density Estimator (KDE)-based algorithms advanced to alleviate such systematic errors rely on bin-level smoothing. In the present work, we propose an alternative algorithm that uses smoothing on the probability distribution level. We consider several smoothing functions, both in the probability space and in its frequency space. An experimental approach is used to investigate the effect of such modification on the computation of both the entropy and the AMI. Results show that, to a significant extent, the present method is able to remove systematic errors in computing entropy and AMI. It is also shown that the present algorithm leads to better reconstruction of multivariate time series when AMI is used in conjunction with their independent components.