English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Deep learning systems have been reported to acheive state-of-the-artperformances in many applications, and one of the keys for achieving this isthe existence of well trained classifiers on benchmark datasets which can beused as backbone feature extractors in downstream tasks. As a main-stream lossfunction for training deep neural network (DNN) classifiers, the cross entropyloss can easily lead us to find models which demonstrate severe overfittingbehavior when no other techniques are used for alleviating it such as dataaugmentation. In this paper, we prove that the existing cross entropy lossminimization for training DNN classifiers essentially learns the conditionalentropy of the underlying data distribution of the dataset, i.e., theinformation or uncertainty remained in the labels after revealing the input. Inthis paper, we propose a mutual information learning framework where we trainDNN classifiers via learning the mutual information between the label andinput. Theoretically, we give the population error probability lower bound interms of the mutual information. In addition, we derive the mutual informationlower and upper bounds for a concrete binary classification data model in$\mbR^n$, and also the error probability lower bound in this scenario. Besides,we establish the sample complexity for accurately learning the mutualinformation from empirical data samples drawn from the underlying datadistribution. Empirically, we conduct extensive experiments on severalbenchmark datasets to support our theory. Without whistles and bells, theproposed mutual information learned classifiers (MILCs) acheive far bettergeneralization performances than the state-of-the-art classifiers with animprovement which can exceed more than 10\% in testing accuracy.

Mutual Information Learned Classifiers: an Information-theoretic Viewpoint of Training Deep Learning Classification Systems