English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

LLMs with superior response quality--particularly larger or closed-sourcemodels--often come with higher inference costs, making their deploymentinefficient and costly. Meanwhile, developing foundational LLMs from scratch isbecoming increasingly resource-intensive and impractical for many applications.To address the challenge of balancing quality and cost, we introduce Routoo, anarchitecture designed to optimize the selection of LLMs for specific promptsbased on performance, cost, and efficiency. Routoo provides controllabilityover the trade-off between inference cost and quality, enabling significantreductions in inference costs for a given quality requirement. Routoo comprisestwo key components: a performance predictor and cost-aware selector. Theperformance predictor is a lightweight LLM that estimates the expectedperformance of various underlying LLMs on a given prompt without executingthem. The cost-aware selector module then selects the most suitable model basedon these predictions and constraints such as cost and latency, significantlyreducing inference costs for the same quality. We evaluated Routoo using theMMLU benchmark across 57 domains employing open-source models. Our results showthat Routoo matches the performance of the Mixtral 8x7b model while reducinginference costs by one-third. Additionally, by allowing increased costs, Routoosurpasses Mixtral's accuracy by over 5% at equivalent costs, achieving anaccuracy of 75.9%. When integrating GPT4 into our model pool, Routoo nearlymatches GPT4's performance at half the cost and exceeds it with a 25% costreduction. These outcomes highlight Routoo's potential to significantly reduceinference costs without compromising quality, and even to establish newstate-of-the-art results by leveraging the collective capabilities of multipleLLMs.

Routoo: Learning to Route to Large Language Models Effectively