English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large transformer models can highly improve Answer Sentence Selection (AS2)tasks, but their high computational costs prevent their use in many real-worldapplications. In this paper, we explore the following research question: Howcan we make the AS2 models more accurate without significantly increasing theirmodel complexity? To address the question, we propose a Multiple Heads Studentarchitecture (named CERBERUS), an efficient neural network designed to distillan ensemble of large transformers into a single smaller model. CERBERUSconsists of two components: a stack of transformer layers that is used toencode inputs, and a set of ranking heads; unlike traditional distillationtechnique, each of them is trained by distilling a different large transformerarchitecture in a way that preserves the diversity of the ensemble members. Theresulting model captures the knowledge of heterogeneous transformer models byusing just a few extra parameters. We show the effectiveness of CERBERUS onthree English datasets for AS2; our proposed approach outperforms allsingle-model distillations we consider, rivaling the state-of-the-art large AS2models that have 2.7x more parameters and run 2.5x slower. Code for our modelis available at https://github.com/amazon-research/wqa-cerberus

Ensemble Transformer for Efficient and Accurate Ranking Tasks: an Application to Question Answering Systems