English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This paper presents a novel pre-trained language models (PLM) compressionapproach based on the matrix product operator (short as MPO) from quantummany-body physics. It can decompose an original matrix into central tensors(containing the core information) and auxiliary tensors (with only a smallproportion of parameters). With the decomposed MPO structure, we propose anovel fine-tuning strategy by only updating the parameters from the auxiliarytensors, and design an optimization algorithm for MPO-based approximation overstacked network architectures. Our approach can be applied to the original orthe compressed PLMs in a general way, which derives a lighter network andsignificantly reduces the parameters to be fine-tuned. Extensive experimentshave demonstrated the effectiveness of the proposed approach in modelcompression, especially the reduction in finetuning parameters (91% reductionon average).

Enabling Lightweight Fine-tuning for Pre-trained Language Model Compression based on Matrix Product Operators