English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Transformers are widely used in NLP tasks. However, current approaches toleveraging transformers to understand language expose one weak spot: Numberunderstanding. In some scenarios, numbers frequently occur, especially insemi-structured data like tables. But current approaches to rich-number taskswith transformer-based language models abandon or lose some of the numeracyinformation - e.g., breaking numbers into sub-word tokens - which leads to manynumber-related errors. In this paper, we propose the LUNA framework whichimproves the numerical reasoning and calculation capabilities oftransformer-based language models. With the number plugin of NumTok and NumBed,LUNA represents each number as a whole to model input. With numberpre-training, including regression loss and model distillation, LUNA bridgesthe gap between number and vocabulary embeddings. To the best of our knowledge,this is the first work that explicitly injects numeracy capability intolanguage models using Number Plugins. Besides evaluating toy models on toytasks, we evaluate LUNA on three large-scale transformer models (RoBERTa, BERT,TabBERT) over three different downstream tasks (TATQA, TabFact, CrediTrans),and observe the performances of language models are constantly improved byLUNA. The augmented models also improve the official baseline of TAT-QA (EM:50.15 -> 59.58) and achieve SOTA performance on CrediTrans (F1 = 86.17).

LUNA: Language Understanding with Number Augmentations on Transformers via Number Plugins and Pre-training