English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The Mixture-of-Experts (MoE) approach has demonstrated outstandingscalability in multi-task learning including low-level upstream tasks such asconcurrent removal of multiple adverse weather effects. However, theconventional MoE architecture with parallel Feed Forward Network (FFN) expertsleads to significant parameter and computational overheads that hinder itsefficient deployment. In addition, the naive MoE linear router is suboptimal inassigning task-specific features to multiple experts which limits its furtherscalability. In this work, we propose an efficient MoE architecture with weightsharing across the experts. Inspired by the idea of linear feature modulation(FM), our architecture implicitly instantiates multiple experts via learnableactivation modulations on a single shared expert block. The proposed FeatureModulated Expert (FME) serves as a building block for the novelMixture-of-Feature-Modulation-Experts (MoFME) architecture, which can scale upthe number of experts with low overhead. We further propose anUncertainty-aware Router (UaR) to assign task-specific features to different FMmodules with well-calibrated weights. This enables MoFME to effectively learndiverse expert functions for multiple tasks. The conducted experiments on themulti-deweather task show that our MoFME outperforms the baselines in the imagerestoration quality by 0.1-0.2 dB and achieves SOTA-compatible performancewhile saving more than 72% of parameters and 39% inference time over theconventional MoE counterpart. Experiments on the downstream segmentation andclassification tasks further demonstrate the generalizability of MoFME to realopen-world applications.

Efficient Deweather Mixture-of-Experts with Uncertainty-aware Feature-wise Linear Modulation