English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large Language Models (LLMs) have shown promise in automated code generationbut typically excel only in simpler tasks such as generating standalone codeunits. Real-world software development, however, often involves complex coderepositories (named repo) with complex dependencies and extensivedocumentation. To fill this gap, our research pivots towards evaluating LLMs ina more realistic setting -- real-world repo-level code generation. We introduceCodeAgentBench, a manually curated benchmark for repo-level code generation.This benchmark comprises five high-quality Python projects, encompassing atotal of 101 samples. We assess nine leading LLMs on repo-level tasks andobserve a decline in their performance. To tackle this, we present CodeAgent, anovel LLM-based agent framework that employs external tools for effectiverepo-level code generation. CodeAgent integrates five programming tools,enabling interaction with software artifacts for information retrieval, codesymbol navigation, and code testing. We implement four agent strategies tooptimize these tools' usage. Our experiments on CodeAgentBench show thatCodeAgent enhances LLM performance significantly, with improvements rangingfrom 18.1\% to 250\%. Further tests on the HumanEval benchmark confirmCodeAgent's adaptability and efficacy across various code generation tasks.Notably, CodeAgent outperforms commercial products like Github Copilot,showcasing superior accuracy and efficiency. These results demonstrateCodeAgent's robust capabilities in code generation, highlighting its potentialfor real-world repo-level coding challenges.

CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges