English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Online experimentation platforms abstract away many of the details ofexperimental design, ensuring experimenters do not have to worry aboutsampling, randomisation, subject tracking, data collection, metric definitionand interpretation of results. The recent success and rapid adoption of theseplatforms in the industry might in part be attributed to the ease-of-use theseabstractions provide. Previous authors have pointed out there are commonpitfalls to avoid when running controlled experiments on the web and emphasisedthe need for experts familiar with the entire software stack to be involved inthe process.  In this paper, we argue that these pitfalls and the need to understand theunderlying complexity are not the result of shortcomings specific to existingplatforms which might be solved by better platform design. We postulate thatthey are a direct consequence of what is commonly referred to as "the law ofleaky abstractions". That is, it is an inherent feature of any softwareplatform that details of its implementation leak to the surface, and that incertain situations, the platform's consumers necessarily need to understanddetails of underlying systems in order to make proficient use of it.  We present several examples of this concept, including examples fromliterature, and suggest some possible mitigation strategies that can beemployed to reduce the impact of abstraction leakage. The conceptual frameworkput forward in this paper allows us to explicitly categorize experimentationpitfalls in terms of which specific abstraction is leaking, thereby aidingimplementers and users of these platforms to better understand and tackle thechallenges they face.

Leaky Abstraction In Online Experimentation Platforms: A Conceptual Framework To Categorize Common Challenges