English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We present our submission to the BabyLM challenge, whose goal was to improvethe sample efficiency of language models. We trained an ensemble consisting ofa GPT-2 and small LLaMA models on the developmentally-plausible, 10M-wordBabyLM dataset, then distilled it into a small, 58M-parameter LLaMA model,which exceeds in performance both of its teachers as well as a similar modeltrained without distillation. This suggests that distillation can not onlyretain the full performance of the teacher model when the latter is trained ona sufficiently small dataset; it can exceed it, and lead to significantlybetter performance than direct training.

Baby Llama: knowledge distillation from an ensemble of teachers trained on a small dataset with no performance penalty