English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Code generation is a longstanding challenge, aiming to generate a codesnippet based on a natural language description. Usually, expensive text-codepaired data is essential for training a code generation model. Recently, thanksto the success of pre-training techniques, large language models are trained onlarge-scale unlabelled code corpora and perform well in code generation. Inthis paper, we investigate how to leverage an unlabelled code corpus to train amodel for library-oriented code generation. Since it is a common practice forprogrammers to reuse third-party libraries, in which case the text-code paireddata are harder to obtain due to the huge number of libraries. We observe thatlibrary-oriented code snippets are more likely to share similar code sketches.Hence, we present CERT with two steps: a sketcher generates the sketch, then agenerator fills the details in the sketch. Both the sketcher and the generatorare continually pre-trained upon a base model using unlabelled data.Furthermore, we craft two benchmarks named PandasEval and NumpyEval to evaluatelibrary-oriented code generation. Experimental results demonstrate theimpressive performance of CERT. For example, it surpasses the base model by anabsolute 15.67% improvement in terms of pass@1 on PandasEval. Our work isavailable at https://github.com/microsoft/PyCodeGPT.

CERT: Continual Pre-Training on Sketches for Library-Oriented Code Generation