English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Implicit models such as Deep Equilibrium Models (DEQs) have garneredsignificant attention in the community for their ability to train infinitelayer models with elegant solution-finding procedures and constant memoryfootprint. However, despite several attempts, these methods are heavilyconstrained by model inefficiency and optimization instability. Furthermore,fair benchmarking across relevant methods for vision tasks is missing. In thiswork, we revisit the line of implicit models and trace them back to theoriginal weight-tied models. Surprisingly, we observe that weight-tied modelsare more effective, stable, as well as efficient on vision tasks, compared tothe DEQ variants. Through the lens of these simple-yet-clean weight-tiedmodels, we further study the fundamental limits in the model capacity of suchmodels and propose the use of distinct sparse masks to improve the modelcapacity. Finally, for practitioners, we offer design guidelines regarding thedepth, width, and sparsity selection for weight-tied models, and demonstratethe generalizability of our insights to other learning paradigms.

Revisiting Implicit Models: Sparsity Trade-offs Capability in Weight-tied Model for Vision Tasks