English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Testing Deep Learning (DL) based systems inherently requires large andrepresentative test sets to evaluate whether DL systems generalise beyond theirtraining datasets. Diverse Test Input Generators (TIGs) have been proposed toproduce artificial inputs that expose issues of the DL systems by triggeringmisbehaviours. Unfortunately, such generated inputs may be invalid, i.e., notrecognisable as part of the input domain, thus providing an unreliable qualityassessment. Automated validators can ease the burden of manually checking thevalidity of inputs for human testers, although input validity is a conceptdifficult to formalise and, thus, automate.  In this paper, we investigate to what extent TIGs can generate valid inputs,according to both automated and human validators. We conduct a large empiricalstudy, involving 2 different automated validators, 220 human assessors, 5different TIGs and 3 classification tasks. Our results show that 84%artificially generated inputs are valid, according to automated validators, buttheir expected label is not always preserved. Automated validators reach a goodconsensus with humans (78% accuracy), but still have limitations when dealingwith feature-rich datasets.

When and Why Test Generators for Deep Learning Produce Invalid Inputs: an Empirical Study