English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Among the many tasks that Large Language Models (LLMs) have revolutionized istext classification. However, existing approaches for applying pretrained LLMsto text classification predominantly rely on using single token outputs fromonly the last layer of hidden states. As a result, they suffer from limitationsin efficiency, task-specificity, and interpretability. In our work, wecontribute an approach that uses all internal representations by employingmultiple pooling strategies on all activation and hidden states. Our novellightweight strategy, Sparsify-then-Classify (STC) first sparsifiestask-specific features layer-by-layer, then aggregates across layers for textclassification. STC can be applied as a seamless plug-and-play module on top ofexisting LLMs. Our experiments on a comprehensive set of models and datasetsdemonstrate that STC not only consistently improves the classificationperformance of pretrained and fine-tuned models, but is also more efficient forboth training and inference, and is more intrinsically interpretable.

Sparsify-then-Classify: From Internal Neurons of Large Language Models To Efficient Text Classifiers