English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Early Exiting is one of the most popular methods to achieve efficientinference. Current early exiting methods adopt the (weighted) sum of the crossentropy loss of all internal classifiers during training, imposing all theseclassifiers to predict all instances correctly. However, during inference, aslong as one internal classifier predicts an instance correctly, it canaccelerate without losing accuracy. Thus, there is a notable gap betweentraining and inference. We propose ConsistentEE, an early exiting method thatis consistent in training and inference. ConsistentEE formulates the earlyexiting process as a reinforcement learning problem. A policy network is addedto decide whether an instance should exit or continue. The training objectiveof ConsistentEE only require each instance to be predicted correctly by oneinternal classifier. Additionally, we introduce the concept Memorize Layer tomeasure the hardness of an instance. We incorporate memorized layer into rewardfunction design, which allows ``easy'' instances to focus more on accelerationwhile ``hard'' instances to focus more on accuracy. Experimental results showthat our method outperforms other baselines on various natural languageunderstanding and generation tasks.

ConsistentEE: A Consistent and Hardness-Guided Early Exiting Method for Accelerating Language Models Inference