English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

As one of the central tasks in machine learning, regression finds lots ofapplications in different fields. An existing common practice for solvingregression problems is the mean square error (MSE) minimization approach or itsregularized variants which require prior knowledge about the models. Recently,Yi et al., proposed a mutual information based supervised learning frameworkwhere they introduced a label entropy regularization which does not require anyprior knowledge. When applied to classification tasks and solved via astochastic gradient descent (SGD) optimization algorithm, their approachachieved significant improvement over the commonly used cross entropy loss andits variants. However, they did not provide a theoretical convergence analysisof the SGD algorithm for the proposed formulation. Besides, applying theframework to regression tasks is nontrivial due to the potentially infinitesupport set of the label. In this paper, we investigate the regression underthe mutual information based supervised learning framework. We first argue thatthe MSE minimization approach is equivalent to a conditional entropy learningproblem, and then propose a mutual information learning formulation for solvingregression problems by using a reparameterization technique. For the proposedformulation, we give the convergence analysis of the SGD algorithm for solvingit in practice. Finally, we consider a multi-output regression data model wherewe derive the generalization performance lower bound in terms of the mutualinformation associated with the underlying data distribution. The result showsthat the high dimensionality can be a bless instead of a curse, which iscontrolled by a threshold. We hope our work will serve as a good starting pointfor further research on the mutual information based regression.

Mutual Information Learned Regressor: an Information-theoretic Viewpoint of Training Regression Systems