English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

On-device learning and efficient fine-tuning enable continuous andprivacy-preserving customization (e.g., locally fine-tuning large languagemodels on personalized data). However, existing training frameworks aredesigned for cloud servers with powerful accelerators (e.g., GPUs, TPUs) andlack the optimizations for learning on the edge, which faces challenges ofresource limitations and edge hardware diversity. We introduce PockEngine: atiny, sparse and efficient engine to enable fine-tuning on various edgedevices. PockEngine supports sparse backpropagation: it prunes the backwardgraph and sparsely updates the model with measured memory saving and latencyreduction while maintaining the model quality. Secondly, PockEngine iscompilation first: the entire training graph (including forward, backward andoptimization steps) is derived at compile-time, which reduces the runtimeoverhead and brings opportunities for graph transformations. PockEngine alsointegrates a rich set of training graph optimizations, thus can furtheraccelerate the training cost, including operator reordering and backendswitching. PockEngine supports diverse applications, frontends and hardwarebackends: it flexibly compiles and tunes models defined inPyTorch/TensorFlow/Jax and deploys binaries to mobile CPU/GPU/DSPs. Weevaluated PockEngine on both vision models and large language models.PockEngine achieves up to 15 $\times$ speedup over off-the-shelf TensorFlow(Raspberry Pi), 5.6 $\times$ memory saving back-propagation (Jetson AGX Orin).Remarkably, PockEngine enables fine-tuning LLaMav2-7B on NVIDIA Jetson AGX Orinat 550 tokens/s, 7.9$\times$ faster than the PyTorch.

PockEngine: Sparse and Efficient Fine-tuning in a Pocket