English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

With easier access to powerful compute resources, there is a growing trend inthe field of AI for software development to develop larger and larger languagemodels (LLMs) to address a variety of programming tasks. Even LLMs applied totasks from the high-performance computing (HPC) domain are huge in size (e.g.,billions of parameters) and demand expensive compute resources for training. Wefound this design choice confusing - why do we need large LLMs trained onnatural languages and programming languages unrelated to HPC for HPC-specifictasks? In this line of work, we aim to question design choices made by existingLLMs by developing smaller LLMs for specific domains - we call themdomain-specific LLMs. Specifically, we start off with HPC as a domain andpropose a novel tokenizer named Tokompiler, designed specifically forpreprocessing code in HPC and compilation-centric tasks. Tokompiler leveragesknowledge of language primitives to generate language-oriented tokens,providing a context-aware understanding of code structure while avoiding humansemantics attributed to code structures completely. We applied Tokompiler topre-train two state-of-the-art models, SPT-Code and Polycoder, for a Fortrancode corpus mined from GitHub. We evaluate the performance of these modelsagainst the conventional LLMs. Results demonstrate that Tokompilersignificantly enhances code completion accuracy and semantic understandingcompared to traditional tokenizers in normalized-perplexity tests, down to ~1perplexity score. This research opens avenues for further advancements indomain-specific LLMs, catering to the unique demands of HPC and compilationtasks.

Scope is all you need: Transforming LLMs for HPC Code