English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Using large pre-trained models for image recognition tasks is becomingincreasingly common owing to the well acknowledged success of recent modelslike vision transformers and other CNN-based models like VGG and Resnet. Thehigh accuracy of these models on benchmark tasks has translated into theirpractical use across many domains including safety-critical applications likeautonomous driving and medical diagnostics. Despite their widespread use, imagemodels have been shown to be fragile to changes in the operating environment,bringing their robustness into question. There is an urgent need for methodsthat systematically characterise and quantify the capabilities of these modelsto help designers understand and provide guarantees about their safety androbustness. In this paper, we propose Vision Checklist, a framework aimed atinterrogating the capabilities of a model in order to produce a report that canbe used by a system designer for robustness evaluations. This frameworkproposes a set of perturbation operations that can be applied on the underlyingdata to generate test samples of different types. The perturbations reflectpotential changes in operating environments, and interrogate various propertiesranging from the strictly quantitative to more qualitative. Our framework isevaluated on multiple datasets like Tinyimagenet, CIFAR10, CIFAR100 andCamelyon17 and for models like ViT and Resnet. Our Vision Checklist proposes aspecific set of evaluations that can be integrated into the previously proposedconcept of a model card. Robustness evaluations like our checklist will becrucial in future safety evaluations of visual perception modules, and beuseful for a wide range of stakeholders including designers, deployers, andregulators involved in the certification of these systems. Source code ofVision Checklist would be open for public use.

Vision Checklist: Towards Testable Error Analysis of Image Models to Help System Designers Interrogate Model Capabilities