English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Web applications are increasingly becoming the primary platform for AIservice delivery, making in-browser deep learning (DL) inference moreprominent. However, current in-browser inference systems fail to effectivelyutilize advanced web programming techniques and customize kernels for variousclient devices, leading to suboptimal performance.  To address the issues, this paper presents the first in-browser inferencesystem, nn-JIT.web, which enables just-in-time (JIT) auto-generation ofoptimized kernels for both CPUs and GPUs during inference. The system achievesthis by using two novel web programming techniques that can significantlyreduce kernel generation time, compared to other tensor compilers such as TVM,while maintaining or even improving performance. The first technique,Tensor-Web Compiling Co-Design, lowers compiling costs by unifying tensor andweb compiling and eliminating redundant and ineffective compiling passes. Thesecond technique, Web-Specific Lite Kernel Optimization Space Design, reduceskernel tuning costs by focusing on web programming requirements and efficienthardware resource utilization, limiting the optimization space to only dozens.  nn-JIT.web is evaluated for modern transformer models on a range of clientdevices, including the mainstream CPUs and GPUs from ARM, Intel, AMD andNvidia. Results show that nn-JIT.web can achieve up to 8.2x faster within 30seconds compared to the baselines across various models.

Accelerating In-Browser Deep Learning Inference on Diverse Edge Clients through Just-in-Time Kernel Optimizations