English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Code datasets, often collected from diverse and uncontrolled sources such asGitHub, potentially suffer from quality issues, thereby affecting theperformance and training efficiency of Large Language Models (LLMs) optimizedfor code generation. Previous studies demonstrated the benefit of usingembedding spaces for data pruning, but they mainly focused on duplicate removalor increasing variety, and in other modalities, such as images. Our workfocuses on using embeddings to identify and remove "low-quality" code data.First, we explore features of "low-quality" code in embedding space, throughthe use of synthetic corruptions. Armed with this knowledge, we devise novelpruning metrics that operate in embedding space to identify and removelow-quality entries in the Stack dataset. We demonstrate the benefits of thissynthetic corruption informed pruning (SCIP) approach on the well-establishedHumanEval and MBPP benchmarks, outperforming existing embedding-based methods.Importantly, we achieve up to a 3% performance improvement over no pruning,thereby showing the promise of insights from synthetic corruptions for datapruning.

Decoding Data Quality via Synthetic Corruptions: Embedding-guided Pruning of Code Data