English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Compositional generalization benchmarks for semantic parsing seek to assesswhether models can accurately compute meanings for novel sentences, butoperationalize this in terms of logical form (LF) prediction. This raises theconcern that semantically irrelevant details of the chosen LFs could shapemodel performance. We argue that this concern is realized for the COGSbenchmark. COGS poses generalization splits that appear impossible forpresent-day models, which could be taken as an indictment of those models.However, we show that the negative results trace to incidental features of COGSLFs. Converting these LFs to semantically equivalent ones and factoring outcapabilities unrelated to semantic interpretation, we find that even baselinemodels get traction. A recent variable-free translation of COGS LFs suggestssimilar conclusions, but we observe this format is not semantically equivalent;it is incapable of accurately representing some COGS meanings. These findingsinform our proposal for ReCOGS, a modified version of COGS that comes closer toassessing the target semantic capabilities while remaining very challenging.Overall, our results reaffirm the importance of compositional generalizationand careful benchmark task design.

ReCOGS: How Incidental Details of a Logical Form Overshadow an Evaluation of Semantic Interpretation