English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In recent years, a variety of gradient-based methods have been developed tosolve Bi-Level Optimization (BLO) problems in machine learning and computervision areas. However, the theoretical correctness and practical effectivenessof these existing approaches always rely on some restrictive conditions (e.g.,Lower-Level Singleton, LLS), which could hardly be satisfied in real-worldapplications. Moreover, previous literature only proves theoretical resultsbased on their specific iteration strategies, thus lack a general recipe touniformly analyze the convergence behaviors of different gradient-based BLOs.In this work, we formulate BLOs from an optimistic bi-level viewpoint andestablish a new gradient-based algorithmic framework, named Bi-level DescentAggregation (BDA), to partially address the above issues. Specifically, BDAprovides a modularized structure to hierarchically aggregate both the upper-and lower-level subproblems to generate our bi-level iterative dynamics.Theoretically, we establish a general convergence analysis template and derivea new proof recipe to investigate the essential theoretical properties ofgradient-based BLO methods. Furthermore, this work systematically explores theconvergence behavior of BDA in different optimization scenarios, i.e.,considering various solution qualities (i.e., global/local/stationary solution)returned from solving approximation subproblems. Extensive experiments justifyour theoretical results and demonstrate the superiority of the proposedalgorithm for hyper-parameter optimization and meta-learning tasks. Source codeis available at https://github.com/vis-opt-group/BDA.

A General Descent Aggregation Framework for Gradient-based Bi-level Optimization