Open AccessLARGE‐SCALE DATA VISUALIZATION WITH MISSING VALUES
Author(s) -
Sergiy Popov
Publication year - 2006
Publication title -
technological and economic development of economy
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.634
H-Index - 47
eISSN - 2029-4921
pISSN - 2029-4913
DOI - 10.3846/13928619.2006.9637721
Subject(s) - missing data , artificial neural network , computer science , dimensionality reduction , data set , bottleneck , visualization , data mining , set (abstract data type) , artificial intelligence , curse of dimensionality , imputation (statistics) , scale (ratio) , pattern recognition (psychology) , machine learning , physics , quantum mechanics , programming language , embedded system
Visualization of large‐scale data inherently requires dimensionality reduction to 1D, 2D, or 3D space. Autoassociative neural networks with a bottleneck layer are commonly used as a nonlinear dimensionality reduction technique. However, many real‐world problems suffer from incomplete data sets, i.e. some values can be missing. Common methods dealing with missing data include the deletion of all cases with missing values from the data set or replacement with mean or “normal” values for specific variables. Such methods are appropriate when just a few values are missing. But in the case when a substantial portion of data is missing, these methods can significantly bias the results of modeling. To overcome this difficulty, we propose a modified learning procedure for the autoassociative neural network that directly takes the missing values into account. The outputs of the trained network may be used for substitution of the missing values in the original data set.