English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In the last decade, it has been shown that many hard AI tasks, especially inNLP, can be naturally modeled as extreme classification problems leading toimproved precision. However, such models are prohibitively expensive to traindue to the memory blow-up in the last layer. For example, a reasonable softmaxlayer for the dataset of interest in this paper can easily reach well beyond100 billion parameters (>400 GB memory). To alleviate this problem, we presentMerged-Average Classifiers via Hashing (MACH), a generic K-classificationalgorithm where memory provably scales at O(logK) without any strong assumptionon the classes. MACH is subtly a count-min sketch structure in disguise, whichuses universal hashing to reduce classification with a large number of classesto few embarrassingly parallel and independent classification tasks with asmall (constant) number of classes. MACH naturally provides a technique forzero communication model parallelism. We experiment with 6 datasets; somemulticlass and some multilabel, and show consistent improvement over respectivestate-of-the-art baselines. In particular, we train an end-to-end deepclassifier on a private product search dataset sampled from Amazon SearchEngine with 70 million queries and 49.46 million products. MACH outperforms, bya significant margin,the state-of-the-art extreme classification modelsdeployed on commercial search engines: Parabel and dense embedding models. Ourlargest model has 6.4 billion parameters and trains in less than 35 hours on asingle p3.16x machine. Our training times are 7-10x faster, and our memoryfootprints are 2-4x smaller than the best baselines. This training time is alsosignificantly lower than the one reported by Google's mixture of experts (MoE)language model on a comparable model size and hardware.

Extreme Classification in Log Memory using Count-Min Sketch: A Case Study of Amazon Search with 50M Products