English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Deep equilibrium networks (DEQs) are a new class of models that eschewstraditional depth in favor of finding the fixed point of a single nonlinearlayer. These models have been shown to achieve performance competitive with thestate-of-the-art deep networks while using significantly less memory. Yet theyare also slower, brittle to architectural choices, and introduce potentialinstability to the model. In this paper, we propose a regularization scheme forDEQ models that explicitly regularizes the Jacobian of the fixed-point updateequations to stabilize the learning of equilibrium models. We show that thisregularization adds only minimal computational cost, significantly stabilizesthe fixed-point convergence in both forward and backward passes, and scaleswell to high-dimensional, realistic domains (e.g., WikiText-103 languagemodeling and ImageNet classification). Using this method, we demonstrate, forthe first time, an implicit-depth model that runs with approximately the samespeed and level of performance as popular conventional deep networks such asResNet-101, while still maintaining the constant memory footprint andarchitectural simplicity of DEQs. Code is available athttps://github.com/locuslab/deq .

Stabilizing Equilibrium Models by Jacobian Regularization