English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Recently, Visual Programming (VProg) has emerged as a significant frameworkfor visual reasoning (VR) tasks due to its interpretability and cross-taskgenerality. However, even with invoking powerful pre-trained Vision-Languagemodels (VLMs) as visual sub-modules, the performance of VProg on specific VRtasks is markedly inferior compared to well-trained task-specific networks.Although invoking task-specific models can further enhance the performance ofVProg on specific VR tasks, it greatly diminishes the cross-task generalizationability of VProg. Besides, the non-differentiable nature of VProg preventsdirect fine-tuning on specific VR tasks for further performance improvement.Attempt to address these issues, we propose SDVP, a Stepwise Distillationlearning strategy for non-differentiable VPorg across various VR tasks.Specifically, our SDVP stepwise distills the capabilities of existing,well-trained small task-specific models for decomposed visual sub-tasks inVProg into the much larger VLMs invoked by corresponding visual sub-modules.Besides, distilling the knowledge of little-size task-specific models intopre-trained larger VLMs rather than replacing them helps keep the cross-taskabilities of VProgs. Extensive and comprehensive experimental results ondifferent VProg frameworks demonstrate that our SDVP obtains significantperformance gains on specific VR benchmarks, i.e., GQA (+2.4\%) and NLVRv2(+6.2\%) for VisProg and GQA (+6.5\%) and NLVRv2 (+4.0\%) for ViperGPT, andalso maintains a promising performance for VProg on unseen and previous VRtasks.

A Stepwise Distillation Learning Strategy for Non-differentiable Visual Programming Frameworks on Visual Reasoning Tasks