English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Deep Neural Networks (DNN) are typically tested for accuracy relying on a setof unlabelled real world data (operational dataset), from which a subset isselected, manually labelled and used as test suite. This subset is required tobe small (due to manual labelling cost) yet to faithfully represent theoperational context, with the resulting test suite containing roughly the sameproportion of examples causing misprediction (i.e., failing test cases) as theoperational dataset. However, while testing to estimate accuracy, it isdesirable to also learn as much as possible from the failing tests in theoperational dataset, since they inform about possible bugs of the DNN. A smartsampling strategy may allow to intentionally include in the test suite manyexamples causing misprediction, thus providing this way more valuable inputsfor DNN improvement while preserving the ability to get trustworthy unbiasedestimates. This paper presents a test selection technique (DeepEST) thatactively looks for failing test cases in the operational dataset of a DNN, withthe goal of assessing the DNN expected accuracy by a small and ''informative''test suite (namely with a high number of mispredictions) for subsequent DNNimprovement. Experiments with five subjects, combining four DNN models andthree datasets, are described. The results show that DeepEST provides DNNaccuracy estimates with precision close to (and often better than) those ofexisting sampling-based DNN testing techniques, while detecting from 5 to 30times more mispredictions, with the same test suite size.

Operation is the hardest teacher: estimating DNN accuracy looking for mispredictions