English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

We present a novel notion of outlier, called the Concentration Free OutlierFactor, or CFOF. As a main contribution, we formalize the notion ofconcentration of outlier scores and theoretically prove that CFOF does notconcentrate in the Euclidean space for any arbitrary large dimensionality. Tothe best of our knowledge, there are no other proposals of data analysismeasures related to the Euclidean distance for which it has been providedtheoretical evidence that they are immune to the concentration effect. Wedetermine the closed form of the distribution of CFOF scores in arbitrarilylarge dimensionalities and show that the CFOF score of a point depends on itssquared norm standard score and on the kurtosis of the data distribution, thusproviding a clear and statistically founded characterization of this notion.Moreover, we leverage this closed form to provide evidence that the definitiondoes not suffer of the hubness problem affecting other measures. We prove thatthe number of CFOF outliers coming from each cluster is proportional to clustersize and kurtosis, a property that we call semi-locality. We determine thatsemi-locality characterizes existing reverse nearest neighbor-based outlierdefinitions, thus clarifying the exact nature of their observed local behavior.We also formally prove that classical distance-based and density-based outliersconcentrate both for bounded and unbounded sample sizes and for fixed andvariable values of the neighborhood parameter. We introduce the fast-CFOFalgorithm for detecting outliers in large high-dimensional dataset. Thealgorithm has linear cost, supports multi-resolution analysis, and isembarrassingly parallel. Experiments highlight that the technique is able toefficiently process huge datasets and to deal even with large values of theneighborhood parameter, to avoid concentration, and to obtain excellentaccuracy.

CFOF: A Concentration Free Measure for Anomaly Detection