English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Artificial intelligence (AI) for software engineering (SE) tasks has recentlyachieved promising performance. In this paper, we investigate to what extentthe pre-trained language model truly understands those SE tasks such as codesearch, code summarization, etc. We conduct a comprehensive empirical study ona board set of AI for SE (AI4SE) tasks by feeding them with variant inputs: 1)with various masking rates and 2) with sufficient input subset method. Then,the trained models are evaluated on different SE tasks, including code search,code summarization, and duplicate bug report detection. Our experimentalresults show that pre-trained language models are insensitive to the giveninput, thus they achieve similar performance in these three SE tasks. We referto this phenomenon as overinterpretation, where a model confidently makes adecision without salient features, or where a model finds some irrelevantrelationships between the final decision and the dataset. Our studyinvestigates two approaches to mitigate the overinterpretation phenomenon:whole word mask strategy and ensembling. To the best of our knowledge, we arethe first to reveal this overinterpretation phenomenon to the AI4SE community,which is an important reminder for researchers to design the input for themodels and calls for necessary future work in understanding and implementingAI4SE tasks.

Do Pre-trained Language Models Indeed Understand Software Engineering Tasks?