English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

This paper asks whether current self-supervised learning methods, ifsufficiently scaled up, would be able to reach human-level visual objectrecognition capabilities with the same type and amount of visual experiencehumans learn from. Previous work on this question only considered the scalingof data size. Here, we consider the simultaneous scaling of data size, modelsize, and image resolution. We perform a scaling experiment with visiontransformers up to 633M parameters in size (ViT-H/14) trained with up to 5Khours of human-like video data (long, continuous, mostly egocentric videos)with image resolutions of up to 476x476 pixels. The efficiency of maskedautoencoders (MAEs) as a self-supervised learning algorithm makes it possibleto run this scaling experiment on an unassuming academic budget. We find thatit is feasible to reach human-level object recognition capacity at sub-humanscales of model size, data size, and image size, if these factors are scaled upsimultaneously. To give a concrete example, we estimate that a 2.5B parameterViT model trained with 20K hours (2.3 years) of human-like video data with aspatial resolution of 952x952 pixels should be able to reach roughlyhuman-level accuracy on ImageNet. Human-level competence is thus achievable fora fundamental perceptual capability from human-like perceptual experience(human-like in both amount and type) with extremely generic learning algorithmsand architectures and without any substantive inductive biases.

Scaling may be all you need for achieving human-level object recognition capacity with human-like visual experience