English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Learning from imbalanced data is among the most challenging areas incontemporary machine learning. This becomes even more difficult when consideredthe context of big data that calls for dedicated architectures capable ofhigh-performance processing. Apache Spark is a highly efficient and populararchitecture, but it poses specific challenges for algorithms to be implementedfor it. While oversampling algorithms are an effective way for handling classimbalance, they have not been designed for distributed environments. In thispaper, we propose a holistic look on oversampling algorithms for imbalanced bigdata. We discuss the taxonomy of oversampling algorithms and their mechanismsused to handle skewed class distributions. We introduce a Spark library with 14state-of-the-art oversampling algorithms implemented and evaluate theirefficacy via extensive experimental study. Using binary and multi-class massivedata sets, we analyze the effectiveness of oversampling algorithms and theirrelationships with different types of classifiers. We evaluate the trade-offbetween accuracy and time complexity of oversampling algorithms, as well astheir scalability when increasing the size of data. This allows us to gaininsight into the usefulness of specific components of oversampling algorithmsfor big data, as well as formulate guidelines and recommendations for designingfuture resampling approaches for massive imbalanced data. Our library can bedownloaded from https://github.com/fsleeman/spark-class-balancing.git.

Imbalanced Big Data Oversampling: Taxonomy, Algorithms, Software, Guidelines and Future Directions