English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Code representation learning, which aims to encode the semantics of sourcecode into distributed vectors, plays an important role in recentdeep-learning-based models for code intelligence. Recently, many pre-trainedlanguage models for source code (e.g., CuBERT and CodeBERT) have been proposedto model the context of code and serve as a basis for downstream codeintelligence tasks such as code search, code clone detection, and programtranslation. Current approaches typically consider the source code as a plainsequence of tokens, or inject the structure information (e.g., AST anddata-flow) into the sequential model pre-training. To further explore theproperties of programming languages, this paper proposes SynCoBERT, asyntax-guided multi-modal contrastive pre-training approach for better coderepresentations. Specially, we design two novel pre-training objectivesoriginating from the symbolic and syntactic properties of source code, i.e.,Identifier Prediction (IP) and AST Edge Prediction (TEP), which are designed topredict identifiers, and edges between two nodes of AST, respectively.Meanwhile, to exploit the complementary information in semantically equivalentmodalities (i.e., code, comment, AST) of the code, we propose a multi-modalcontrastive learning strategy to maximize the mutual information amongdifferent modalities. Extensive experiments on four downstream tasks related tocode intelligence show that SynCoBERT advances the state-of-the-art with thesame pre-training corpus and model size.

SynCoBERT: Syntax-Guided Multi-Modal Contrastive Pre-Training for Code Representation