English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Screen recordings of mobile apps are a popular and readily available way forusers to share how they interact with apps, such as in online tutorial videos,user reviews, or as attachments in bug reports. Unfortunately, both people andsystems can find it difficult to reproduce touch-driven interactions from videopixel data alone. In this paper, we introduce an approach to extract and replayuser interactions in videos of mobile apps, using only pixel information invideo frames. To identify interactions, we apply heuristic-based imageprocessing and convolutional deep learning to segment screen recordings,classify the interaction in each segment, and locate the interaction point. Toreplay interactions on another device, we match elements on app screens usingUI element detection. We evaluate the feasibility of our pixel-based approachusing two datasets: the Rico mobile app dataset and a new dataset of 64 appswith both iOS and Android versions. We find that our end-to-end approach cansuccessfully replay a majority of interactions (iOS--84.1%, Android--78.4%) ondifferent devices, which is a step towards supporting a variety of scenarios,including automatically annotating interactions in existing videos, automatedUI testing, and creating interactive app tutorials.

Extracting Replayable Interactions from Videos of Mobile App Usage