English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Dynamic early exiting aims to accelerate the inference of pre-trainedlanguage models (PLMs) by emitting predictions in internal layers withoutpassing through the entire model. In this paper, we empirically analyze theworking mechanism of dynamic early exiting and find that it faces a performancebottleneck under high speed-up ratios. On one hand, the PLMs' representationsin shallow layers lack high-level semantic information and thus are notsufficient for accurate predictions. On the other hand, the exiting decisionsmade by internal classifiers are unreliable, leading to wrongly emitted earlypredictions. We instead propose a new framework for accelerating the inferenceof PLMs, CascadeBERT, which dynamically selects proper-sized and completemodels in a cascading manner, providing comprehensive representations forpredictions. We further devise a difficulty-aware objective, encouraging themodel to output the class probability that reflects the real difficulty of eachinstance for a more reliable cascading mechanism. Experimental results showthat CascadeBERT can achieve an overall 15\% improvement under 4$\times$speed-up compared with existing dynamic early exiting methods on sixclassification tasks, yielding more calibrated and accurate predictions.

CascadeBERT: Accelerating Inference of Pre-trained Language Models via Calibrated Complete Models Cascade