English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

With the growing popularity of Large Language Models (e.g. GitHub Copilot,ChatGPT, etc.) in software engineers' daily practices, it is important toensure that the code generated by these tools is not only functionally correctbut also free of vulnerabilities. Although LLMs can help developers to be moreproductive, prior empirical studies have shown that LLMs can generate insecurecode. There are two contributing factors to the insecure code generation.First, existing datasets used to evaluate Large Language Models (LLMs) do notadequately represent genuine software engineering tasks sensitive to security.Instead, they are often based on competitive programming challenges orclassroom-type coding tasks. In real-world applications, the code produced isintegrated into larger codebases, introducing potential security risks. There'sa clear absence of benchmarks that focus on evaluating the security of thegenerated code. Second, existing evaluation metrics primarily focus on thefunctional correctness of the generated code while ignoring securityconsiderations. Metrics such as pass@k gauge the probability of obtaining thecorrect code in the top k suggestions. Other popular metrics like BLEU,CodeBLEU, ROUGE, and METEOR similarly emphasize functional accuracy, neglectingsecurity implications. In light of these research gaps, in this paper, wedescribed SALLM, a framework to benchmark LLMs' abilities to generate securecode systematically. This framework has three major components: a novel datasetof security-centric Python prompts, an evaluation environment to test thegenerated code, and novel metrics to evaluate the models' performance from theperspective of secure code generation.

Generate and Pray: Using SALLMS to Evaluate the Security of LLM Generated Code