Open AccessEnd‐to‐end learning interpolation for object tracking in low frame‐rate video
Author(s) -
Liu Liqiang,
Cao Jianzhong
Publication year - 2020
Publication title -
iet image processing
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.401
H-Index - 45
eISSN - 1751-9667
pISSN - 1751-9659
DOI - 10.1049/iet-ipr.2019.0944
Subject(s) - computer science , video tracking , frame rate , analytics , frame (networking) , artificial intelligence , interpolation (computer graphics) , computer vision , bandwidth (computing) , end to end principle , motion interpolation , real time computing , object (grammar) , data mining , block matching algorithm , computer network
In many scenarios, where videos are transmitted through bandwidth‐limited channels for subsequent semantic analytics, the choice of frame rates has to balance between bandwidth constraints and analytics performance. Faced with this practical challenge, this study focuses on enhancing object tracking at low frame rates and proposes a learning Interpolation for tracking framework. This framework embeds an implicit video frame interpolation sub‐network, which is concatenated and jointly trained with another object tracking sub‐network. Once a low frame‐rate video is an input, it is first mapped into a high frame‐rate latent video, based on which the tracker is learned. Novel strategies and loss functions are derived to ensure the effective end‐to‐end optimisation of the authors’ network. On several challenging benchmarks and settings, their method achieves a highly competitive tradeoff between frame rate and tracking accuracy. As is known, the implications of interpolation on semantic video analytics and tracking remain unexplored, and the authors expect their method to find many applications in mobile embedded vision, Internet of Things and edge computing.