English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

High-dimensional models often have a large memory footprint and must bequantized after training before being deployed on resource-constrained edgedevices for inference tasks. In this work, we develop an information-theoreticframework for the problem of quantizing a linear regressor learned fromtraining data $(\mathbf{X}, \mathbf{y})$, for some underlying statisticalrelationship $\mathbf{y} = \mathbf{X}\boldsymbol{\theta} + \mathbf{v}$. Thelearned model, which is an estimate of the latent parameter$\boldsymbol{\theta} \in \mathbb{R}^d$, is constrained to be representableusing only $Bd$ bits, where $B \in (0, \infty)$ is a pre-specified budget and$d$ is the dimension. We derive an information-theoretic lower bound for theminimax risk under this setting and propose a matching upper bound usingrandomized embedding-based algorithms which is tight up to constant factors.The lower and upper bounds together characterize the minimum thresholdbit-budget required to achieve a performance risk comparable to the unquantizedsetting. We also propose randomized Hadamard embeddings that arecomputationally efficient and are optimal up to a mild logarithmic factor ofthe lower bound. Our model quantization strategy can be generalized and we showits efficacy by extending the method and upper-bounds to two-layer ReLU neuralnetworks for non-linear regression. Numerical simulations show the improvedperformance of our proposed scheme as well as its closeness to the lower bound.

Minimax Optimal Quantization of Linear Models: Information-Theoretic Limits and Efficient Algorithms