English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We report our experience of using failure symptoms, such as error messages orstack traces, to identify flaky test failures in a Continuous Integration (CI)pipeline for a large industrial software system, SAP HANA. Although failuresymptoms are commonly used to identify similar failures, they have notpreviously been employed to detect flaky test failures. Our hypothesis is thatflaky failures will exhibit symptoms distinct from those of non-flaky failures.Consequently, we can identify recurring flaky failures, without rerunning thetests, by matching the failure symptoms to those of historical flaky runs. Thiscan significantly reduce the need for test reruns, ultimately resulting infaster delivery of test results to developers. To facilitate the process ofmatching flaky failures across different execution instances, we abstract newertest failure symptoms before matching them to the known patterns of flakyfailures, inspired by previous research in the fields of failure deduplicationand log analysis. We evaluate our symptom-based flakiness detection methodusing actual failure symptoms gathered from CI data of SAP HANA during asix-month period. Our method shows the potential of using failure symptoms toidentify recurring flaky failures, achieving a precision of at least 96%, whilesaving approximately 58% of the machine time compared to the traditional rerunstrategy. Analysis of the false positives and the feedback from developersunderscore the importance of having descriptive and informative failuresymptoms for both the effective deployment of this symptom-based approach andthe debugging of flaky tests.

Just-in-Time Flaky Test Detection via Abstracted Failure Symptom Matching