English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Many companies rely on APIs of managed AI models such as OpenAI's GPT-4 tocreate AI-enabled experiences in their products. Along with the benefits ofease of use and shortened time to production, this reliance on proprietary APIshas downsides in terms of model control, performance reliability, up-timepredictability, and cost. At the same time, there has been a flurry of opensource small language models (SLMs) that have been made available forcommercial use. However, their readiness to replace existing capabilitiesremains unclear, and a systematic approach to test these models is not readilyavailable. In this paper, we present a systematic evaluation methodology for,and characterization of, modern open source SLMs and their trade-offs whenreplacing a proprietary LLM APIs for a real-world product feature. We havedesigned SLaM, an automated analysis tool that enables the quantitative andqualitative testing of product features utilizing arbitrary SLMs. Using SLaM,we examine both the quality and the performance characteristics of modern SLMsrelative to an existing customer-facing OpenAI-based implementation. We findthat across 9 SLMs and 29 variants, we observe competitive quality-of-resultsfor our use case, significant performance consistency improvement, and a costreduction of 5x-29x when compared to OpenAI GPT-4.

A Trade-off Analysis of Replacing Proprietary LLMs with Open Source SLMs in Production