English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

In Recurrent Neural Networks (RNNs), encoding information in a suboptimal orerroneous way can impact the quality of representations based on later elementsin the sequence and subsequently lead to wrong predictions and a worse modelperformance. In humans, challenging cases like garden path sentences (aninstance of this being the infamous "The horse raced past the barn fell") canlead their language understanding astray. However, they are still able tocorrect their representation accordingly and recover when new information isencountered. Inspired by this, I propose an augmentation to standard RNNs inform of a gradient-based correction mechanism: This way I hope to enable suchmodels to dynamically adapt their inner representation of a sentence, adding away to correct deviations as soon as they occur. This could therefore lead tomore robust models using more flexible representations, even during inferencetime.  I conduct different experiments in the context of language modeling, wherethe impact of using such a mechanism is examined in detail. To this end, I lookat modifications based on different kinds of time-dependent error signals andhow they influence the model performance. Furthermore, this work contains astudy of the model's confidence in its predictions during training and forchallenging test samples and the effect of the manipulation thereof. Lastly, Ialso study the difference in behavior of these novel models compared to astandard LSTM baseline and investigate error cases in detail to identify pointsof future research. I show that while the proposed approach comes withpromising theoretical guarantees and an appealing intuition, it is only able toproduce minor improvements over the baseline due to challenges in its practicalapplication and the efficacy of the tested model variants.

Recoding latent sentence representations -- Dynamic gradient-based activation modification in RNNs