English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The reasoning capabilities of LLM (Large Language Model) are widelyacknowledged in recent research, inspiring studies on tool learning andautonomous agents. LLM serves as the "brain" of agent, orchestrating multipletools for collaborative multi-step task solving. Unlike methods invoking toolslike calculators or weather APIs for straightforward tasks, multi-modal agentsexcel by integrating diverse AI models for complex challenges. However, currentmulti-modal agents neglect the significance of model selection: they primarilyfocus on the planning and execution phases, and will only invoke predefinedtask-specific models for each subtask, making the execution fragile. Meanwhile,other traditional model selection methods are either incompatible with orsuboptimal for the multi-modal agent scenarios, due to ignorance ofdependencies among subtasks arising by multi-step reasoning.  To this end, we identify the key challenges therein and propose the$\textit{M}^3$ framework as a plug-in with negligible runtime overhead attest-time. This framework improves model selection and bolsters the robustnessof multi-modal agents in multi-step reasoning. In the absence of suitablebenchmarks, we create MS-GQA, a new dataset specifically designed toinvestigate the model selection challenge in multi-modal agents. Ourexperiments reveal that our framework enables dynamic model selection,considering both user inputs and subtask dependencies, thereby robustifying theoverall reasoning process. Our code and benchmark:https://github.com/LINs-lab/M3.

Towards Robust Multi-Modal Reasoning via Model Selection