English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In recent years, the research on empirical software engineering that usesqualitative data analysis (e.g., cases studies, interview surveys, and groundedtheory studies) is increasing. However, most of this research does not deepinto the reliability and validity of findings, specifically in the reliabilityof coding in which these methodologies rely on, despite there exist a varietyof statistical techniques known as Inter-Coder Agreement (ICA) for analyzingconsensus in team coding. This paper aims to establish a novel theoreticalframework that enables a methodological approach for conducting this validityanalysis. This framework is based on a set of coefficients for measuring thedegree of agreement that different coders achieve when judging a common matter.We analyze different reliability coefficients and provide detailed examples ofcalculation, with special attention to Krippendorff's $\alpha$ coefficients. Wesystematically review several variants of Krippendorff's $\alpha$ reported inthe literature and provide a novel common mathematical framework in which allof them are unified through a universal $\alpha$ coefficient. Finally, thispaper provides a detailed guide of the use of this theoretical framework in alarge case study on DevOps culture. We explain how $\alpha$ coefficients arecomputed and interpreted using a widely used software tool for qualitativeanalysis like Atlas.ti. We expect that this work will help empiricalresearchers, particularly in software engineering, to improve the quality andtrustworthiness of their studies.

Reliability in Software Engineering Qualitative Research through Inter-Coder Agreement: A guide using Krippendorff's $\alpha$ & Atlas.ti