English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Tool learning has generated widespread interest as a vital means ofinteraction between Large Language Models (LLMs) and the physical world.Current research predominantly emphasizes LLMs' capacity to utilize tools inwell-structured environments while overlooking their stability when confrontedwith the inevitable noise of the real world. To bridge this gap, we introduceRoTBench, a multi-level benchmark for evaluating the robustness of LLMs in toollearning. Specifically, we establish five external environments, each featuringvarying levels of noise (i.e., Clean, Slight, Medium, Heavy, and Union),providing an in-depth analysis of the model's resilience across three criticalphases: tool selection, parameter identification, and content filling.Experiments involving six widely-used models underscore the urgent necessityfor enhancing the robustness of LLMs in tool learning. For instance, theperformance of GPT-4 even drops significantly from 80.00 to 58.10 when there isno substantial change in manual accuracy. More surprisingly, the noisecorrection capability inherent in the GPT family paradoxically impedes itsadaptability in the face of mild noise. In light of these findings, we proposeRoTTuning, a strategy that enriches the diversity of training environments tobolster the robustness of LLMs in tool learning. The code and data areavailable at https://github.com/Junjie-Ye/RoTBench.

RoTBench: A Multi-Level Benchmark for Evaluating the Robustness of Large Language Models in Tool Learning