English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

A central problem in cognitive science and behavioural neuroscience as wellas in machine learning and artificial intelligence research is to ascertainwhether two or more decision makers (be they brains or algorithms) use the samestrategy. Accuracy alone cannot distinguish between strategies: two systems mayachieve similar accuracy with very different strategies. The need todifferentiate beyond accuracy is particularly pressing if two systems are nearceiling performance, like Convolutional Neural Networks (CNNs) and humans onvisual object recognition. Here we introduce trial-by-trial error consistency,a quantitative analysis for measuring whether two decision making systemssystematically make errors on the same inputs. Making consistent errors on atrial-by-trial basis is a necessary condition for similar processing strategiesbetween decision makers. Our analysis is applicable to compare algorithms withalgorithms, humans with humans, and algorithms with humans. When applying errorconsistency to object recognition we obtain three main findings: (1.)Irrespective of architecture, CNNs are remarkably consistent with one another.(2.) The consistency between CNNs and human observers, however, is little abovewhat can be expected by chance alone -- indicating that humans and CNNs arelikely implementing very different strategies. (3.) CORnet-S, a recurrent modeltermed the "current best model of the primate ventral visual stream", fails tocapture essential characteristics of human behavioural data and behavesessentially like a standard purely feedforward ResNet-50 in our analysis. Takentogether, error consistency analysis suggests that the strategies used by humanand machine vision are still very different -- but we envision ourgeneral-purpose error consistency analysis to serve as a fruitful tool forquantifying future progress.

Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistency