English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Code coverage is a widely used metric for quantifying the extent to whichprogram elements, such as statements or branches, are executed during testing.Calculating code coverage is resource-intensive, requiring code building andexecution with additional overhead for the instrumentation. Furthermore,computing coverage of any snippet of code requires the whole program context.Using Machine Learning to amortize this expensive process could lower the costof code coverage by requiring only the source code context, and the task ofcode coverage prediction can be a novel benchmark for judging the ability ofmodels to understand code. We propose a novel benchmark task called CodeCoverage Prediction for Large Language Models (LLMs). We formalize this task toevaluate the capability of LLMs in understanding code execution by determiningwhich lines of a method are executed by a given test case and inputs. We curateand release a dataset we call COVERAGEEVAL by executing tests and code from theHumanEval dataset and collecting code coverage information. We report theperformance of four state-of-the-art LLMs used for code-related tasks,including OpenAI's GPT-4 and GPT-3.5-Turbo, Google's BARD, and Anthropic'sClaude, on the Code Coverage Prediction task. Finally, we argue that codecoverage as a metric and pre-training data source are valuable for overall LLMperformance on software engineering tasks.

Predicting Code Coverage without Execution