English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

We introduce GLM-130B, a bilingual (English and Chinese) pre-trained languagemodel with 130 billion parameters. It is an attempt to open-source a 100B-scalemodel at least as good as GPT-3 (davinci) and unveil how models of such a scalecan be successfully pre-trained. Over the course of this effort, we facenumerous unexpected technical and engineering challenges, particularly on lossspikes and divergence. In this paper, we introduce the training process ofGLM-130B including its design choices, training strategies for both efficiencyand stability, and engineering efforts. The resultant GLM-130B model offerssignificant outperformance over GPT-3 175B (davinci) on a wide range of popularEnglish benchmarks while the performance advantage is not observed in OPT-175Band BLOOM-176B. It also consistently and significantly outperforms ERNIE TITAN3.0 260B -- the largest Chinese language model -- across related benchmarks.Finally, we leverage a unique scaling property of GLM-130B to reach INT4quantization without post training, with almost no performance loss, making itthe first among 100B-scale models and more importantly, allowing its effectiveinference on 4$\times$RTX 3090 (24G) or 8$\times$RTX 2080 Ti (11G) GPUs, themost affordable GPUs required for using 100B-scale models. The GLM-130B modelweights are publicly accessible and its code, training logs, related toolkit,and lessons learned are open-sourced at\url{https://github.com/THUDM/GLM-130B/}.

GLM-130B: An Open Bilingual Pre-trained Model