English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Modern deep learning is primarily an experimental science, in which empiricaladvances occasionally come at the expense of probabilistic rigor. Here we focuson one such example; namely the use of the categorical cross-entropy loss tomodel data that is not strictly categorical, but rather takes values on thesimplex. This practice is standard in neural network architectures with labelsmoothing and actor-mimic reinforcement learning, amongst others. Drawing onthe recently discovered continuous-categorical distribution, we proposeprobabilistically-inspired alternatives to these models, providing an approachthat is more principled and theoretically appealing. Through carefulexperimentation, including an ablation study, we identify the potential foroutperformance in these models, thereby highlighting the importance of a properprobabilistic treatment, as well as illustrating some of the failure modesthereof.

Uses and Abuses of the Cross-Entropy Loss: Case Studies in Modern Deep Learning