English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large Language Models (LLMs) have successfully been applied to codegeneration tasks, raising the question of how well these models understandprogramming. Typical programming languages have invariances and equivariancesin their semantics that human programmers intuitively understand and exploit,such as the (near) invariance to the renaming of identifiers. We show that LLMsnot only fail to properly generate correct Python code when default functionnames are swapped, but some of them even become more confident in theirincorrect predictions as the model size increases, an instance of the recentlydiscovered phenomenon of Inverse Scaling, which runs contrary to the commonlyobserved trend of increasing prediction quality with increasing model size. Ourfindings indicate that, despite their astonishing typical-case performance,LLMs still lack a deep, abstract understanding of the content they manipulate,making them unsuitable for tasks that statistically deviate from their trainingdata, and that mere scaling is not enough to achieve such capability.

The Larger They Are, the Harder They Fail: Language Models do not Recognize Identifier Swaps in Python