English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Recent breakthroughs in pre-trained code models, such as CodeBERT and Codex,have shown their superior performance in various downstream tasks. Thecorrectness and unambiguity of API usage among these code models are crucialfor achieving desirable program functionalities, requiring them to learnvarious API fully qualified names structurally and semantically. Recent studiesreveal that even state-of-the-art pre-trained code models struggle withsuggesting the correct APIs during code generation. However, the reasons forsuch poor API usage performance are barely investigated. To address thischallenge, we propose using knowledge probing as a means of interpreting codemodels, which uses cloze-style tests to measure the knowledge stored in models.Our comprehensive study examines a code model's capability of understanding APIfully qualified names from two different perspectives: API call and API import.Specifically, we reveal that current code models struggle with understandingAPI names, with pre-training strategies significantly affecting the quality ofAPI name learning. We demonstrate that natural language context can assist codemodels in locating Python API names and generalize Python API name knowledge tounseen data. Our findings provide insights into the limitations andcapabilities of current pre-trained code models, and suggest that incorporatingAPI structure into the pre-training process can improve automated API usage andcode representations. This work provides significance for advancing codeintelligence practices and direction for future studies. All experimentresults, data and source code used in this work are available at\url{https://doi.org/10.5281/zenodo.7902072}.

Pop Quiz! Do Pre-trained Code Models Possess Knowledge of Correct API Names?