English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Due to the labor-intensive nature of manual test oracle construction, variousautomated testing techniques have been proposed to enhance the reliability ofNatural Language Processing (NLP) software. In theory, these techniques mutatean existing test case (e.g., a sentence with its label) and assume thegenerated one preserves an equivalent or similar semantic meaning and thus, thesame label. However, in practice, many of the generated test cases fail topreserve similar semantic meaning and are unnatural (e.g., grammar errors),which leads to a high false alarm rate and unnatural test cases. Our evaluationstudy finds that 44% of the test cases generated by the state-of-the-art (SOTA)approaches are false alarms. These test cases require extensive manual checkingeffort, and instead of improving NLP software, they can even degrade NLPsoftware when utilized in model training. To address this problem, we proposeAEON for Automatic Evaluation Of NLP test cases. For each generated test case,it outputs scores based on semantic similarity and language naturalness. Weemploy AEON to evaluate test cases generated by four popular testing techniqueson five datasets across three typical NLP tasks. The results show that AEONaligns the best with human judgment. In particular, AEON achieves the bestaverage precision in detecting semantic inconsistent test cases, outperformingthe best baseline metric by 10%. In addition, AEON also has the highest averageprecision of finding unnatural test cases, surpassing the baselines by morethan 15%. Moreover, model training with test cases prioritized by AEON leads tomodels that are more accurate and robust, demonstrating AEON's potential inimproving NLP software.

AEON: A Method for Automatic Evaluation of NLP Test Cases