Open AccessAutomated system for identification of data distribution laws by analysis of histogram proximity with sample reduction
Author(s) -
Olha Oliynyk,
Yurii Taranenko
Publication year - 2021
Publication title -
ukraïnsʹkij metrologìčnij žurnal
Language(s) - English
Resource type - Journals
eISSN - 2522-1345
pISSN - 2306-7039
DOI - 10.24027/2306-7039.3.2021.241627
Subject(s) - histogram , kurtosis , sample size determination , computer science , data reduction , principle of maximum entropy , law , entropy (arrow of time) , data mining , mathematics , algorithm , statistics , artificial intelligence , physics , quantum mechanics , political science , image (mathematics)
The error in the identification of the distribution law entails an incorrect assessment of other characteristics (standard deviation, kurtosis, antikurtosis, etc.). The article is devoted to the development of accessible and simple software products for solving problems of identifying distribution laws and determining the optimal size of a data sample.
The paper describes a modified method for identifying the law of data distribution by visual analysis of the proximity of histograms with a reduction in the sample size with software implementation. The method allows choosing the most probable distribution law from a wide base of the set. The essence of the method consists in calculating the entropy coefficient and absolute entropy error for the initial and half data sample, determining the optimal method for processing the histogram using visual analysis of the proximity of histograms, and identifying the data distribution law. The experimental data processing model makes it possible to take into account the statistical properties of real data and can be applied to various arrays, and allows to reduce the sample size required for analysis.
An automated system for identifying the laws of data distribution with a simple and intuitive interface has been developed. The results of the study on real data indicate an increase in the reliability of the identification of the data distribution law.