English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We present the largest and most comprehensive empirical study of pre-trainedvisual representations (PVRs) or visual 'foundation models' for Embodied AI.First, we curate CortexBench, consisting of 17 different tasks spanninglocomotion, navigation, dexterous, and mobile manipulation. Next, wesystematically evaluate existing PVRs and find that none are universallydominant.  To study the effect of pre-training data scale and diversity, we combine over4,000 hours of egocentric videos from 7 different sources (over 5.6M images)and ImageNet to train different-sized vision transformers using MaskedAuto-Encoding (MAE) on slices of this data. Contrary to inferences from priorwork, we find that scaling dataset size and diversity does not improveperformance universally (but does so on average).  Our largest model, named VC-1, outperforms all prior PVRs on average but doesnot universally dominate either. Finally, we show that task or domain-specificadaptation of VC-1 leads to substantial gains, with VC-1 (adapted) achievingcompetitive or superior performance than the best known results on all of thebenchmarks in CortexBench. These models required over 10,000 GPU-hours to trainand can be found on our website for the benefit of the research community.

Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?