English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Pre-trained large language models have significantly improved codegeneration. As these models scale up, there is an increasing need for theoutput to handle more intricate tasks and to be appropriately specialized toparticular domains. Here, we target bioinformatics due to the amount ofspecialized domain knowledge, algorithms, and data operations this disciplinerequires. We present BioCoder, a benchmark developed to evaluate large languagemodels (LLMs) in generating bioinformatics-specific code. BioCoder spans abroad spectrum of the field and covers cross-file dependencies, classdeclarations, and global variables. It incorporates 1026 Python functions and1243 Java methods extracted from GitHub, along with 253 examples from theRosalind Project, all pertaining to bioinformatics. Using topic modeling weshow that overall coverage of the included code is representative of the fullspectrum of bioinformatics calculations. BioCoder incorporates a fuzz-testingframework for evaluation. We have applied it to evaluate many models includingInCoder, CodeGen, CodeGen2, SantaCoder, StarCoder, StarCoder+, InstructCodeT5+,GPT-3.5, and GPT-4. Furthermore, we finetuned StarCoder, demonstrating how ourdataset can effectively enhance the performance of LLMs on our benchmark (by>15% in terms of Pass@K in certain prompt configurations and always >3%). Theresults highlight two key aspects of successful models: (1) Successful modelsaccommodate a long prompt (> ~2600 tokens) with full context, for functionaldependencies. (2) They contain specific domain knowledge of bioinformatics,beyond just general coding knowledge. This is evident from the performance gainof GPT-3.5/4 compared to the smaller models on the benchmark (50% vs up to~25%). Our dataset, benchmark, Docker images, and scripts required for testingare all available at https://github.com/gersteinlab/biocoder.

BioCoder: A Benchmark for Bioinformatics Code Generation with Contextual Pragmatic Knowledge