English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Serving Large Language Models (LLMs) is critical for AI-powered applicationsbut demands substantial computational resources, particularly in memorybandwidth and computational throughput. Low-precision computation has emergedas a key technique to improve efficiency while reducing resource consumption.Existing approaches for generating low-precision kernels are limited to weightbit widths that are powers of two and suffer from suboptimal performance due tohigh-level GPU programming abstractions. These abstractions restrict criticaloptimizations, such as fine-grained register management and optimized memoryaccess patterns, which are essential for efficient low-precision computations.In this paper, we introduce a virtual machine (VM) designed for General-PurposeGPU (GPGPU) computing, enabling support for low-precision data types witharbitrary bit widths while maintaining GPU programmability. The proposed VMfeatures a thread-block-level programming model, a hierarchical memory space, anovel algebraic layout system, and extensive support for diverse low-precisiondata types. VM programs are compiled into highly efficient GPU programs withautomatic vectorization and instruction selection. Extensive experimentsdemonstrate that our VM efficiently supports a full spectrum of low-precisiondata types, and outperforms state-of-the-art low-precision kernels on theirsupported types. Compared to existing compilers like Triton and Ladder, as wellas hand-optimized kernels such as QuantLLM and Marlin, our VM achievesperformance improvements of 1.75x, 2.61x, 1.29x and 1.03x, respectively.

Tilus: A Virtual Machine for Arbitrary Low-Precision GPGPU Computation in LLM Serving