English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Neural models command state-of-the-art performance across NLP tasks,including ones involving "reasoning". Models claiming to reason about theevidence presented to them should attend to the correct parts of the inputavoiding spurious patterns therein, be self-consistent in their predictionsacross inputs, and be immune to biases derived from their pre-training in anuanced, context-sensitive fashion. {\em Do the prevalent *BERT-family ofmodels do so?} In this paper, we study this question using the problem ofreasoning on tabular data. Tabular inputs are especially well-suited for thestudy -- they admit systematic probes targeting the properties listed above.Our experiments demonstrate that a RoBERTa-based model, representative of thecurrent state-of-the-art, fails at reasoning on the following counts: it (a)ignores relevant parts of the evidence, (b) is over-sensitive to annotationartifacts, and (c) relies on the knowledge encoded in the pre-trained languagemodel rather than the evidence presented in its tabular inputs. Finally,through inoculation experiments, we show that fine-tuning the model onperturbed data does not help it overcome the above challenges.

Is My Model Using The Right Evidence? Systematic Probes for Examining Evidence-Based Tabular Reasoning