English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

While large pretrained Transformer models have proven highly capable attackling natural language tasks, handling long sequence inputs continues to bea significant challenge. One such task is long input summarization, whereinputs are longer than the maximum input context of most pretrained models.Through an extensive set of experiments, we investigate what modelarchitectural changes and pretraining paradigms can most efficiently adapt apretrained Transformer for long input summarization. We find that a staggered,block-local Transformer with global encoder tokens strikes a good balance ofperformance and efficiency, and that an additional pretraining phase on longsequences meaningfully improves downstream summarization performance. Based onour findings, we introduce PEGASUS-X, an extension of the PEGASUS model withadditional long input pretraining to handle inputs of up to 16K tokens.PEGASUS-X achieves strong performance on long input summarization taskscomparable with much larger models while adding few additional parameters andnot requiring model parallelism to train.

Investigating Efficiently Extending Transformers for Long Input Summarization