English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

The rapid advancements in large language models (LLMs) have greatly expandedthe potential for automated code-related tasks. Two primary methodologies areused in this domain: prompt engineering and fine-tuning. Prompt engineeringinvolves applying different strategies to query LLMs, like ChatGPT, whilefine-tuning further adapts pre-trained models, such as CodeBERT, by trainingthem on task-specific data. Despite the growth in the area, there remains alack of comprehensive comparative analysis between the approaches for codemodels. In this paper, we evaluate GPT-4 using three prompt engineeringstrategies -- basic prompting, in-context learning, and task-specific prompting-- and compare it against 17 fine-tuned models across three code-related tasks:code summarization, generation, and translation. Our results indicate thatGPT-4 with prompt engineering does not consistently outperform fine-tunedmodels. For instance, in code generation, GPT-4 is outperformed by fine-tunedmodels by 28.3% points on the MBPP dataset. It also shows mixed results forcode translation tasks. Additionally, a user study was conducted involving 27graduate students and 10 industry practitioners. The study revealed that GPT-4with conversational prompts, incorporating human feedback during interaction,significantly improved performance compared to automated prompting.Participants often provided explicit instructions or added context during theseinteractions. These findings suggest that GPT-4 with conversational promptingholds significant promise for automated code-related tasks, whereas fullyautomated prompt engineering without human involvement still requires furtherinvestigation.

Prompt Engineering or Fine-Tuning: An Empirical Assessment of LLMs for Code