English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

The success of ChatGPT has recently attracted numerous efforts to replicateit, with instruction-tuning strategies being a key factor in achievingremarkable results. Instruction-tuning not only significantly enhances themodel's performance and generalization but also makes the model's generatedresults more consistent with human speech patterns. However current researchrarely studies the impact of different amounts of instruction data on modelperformance, especially in the real-world use cases. In this paper we explorethe performance of large language models based on instruction tuning acrossdifferent scales of instruction data. An evaluation dataset consisting of 12major online use cases is constructed in the experiment. With Bloomz-7B1-mt asthe base model, the results show that 1) merely increasing the amount ofinstruction data leads to continuous improvement in tasks such as open-endedgeneration, 2) in tasks such as math and code, the model performance curveremains quite flat while increasing data size. We further analyze the possiblecauses of these phenomena and propose potential future research directions suchas effectively selecting high-quality training data, scaling base models andtraining methods specialized for hard tasks. We will release our training andevaluation datasets, as well as model checkpoints.

Exploring the Impact of Instruction Data Scaling on Large Language Models: An Empirical Study on Real-World Use Cases