Incremental transfer learning for video annotation via grouped heterogeneous sources

Here, the authors focus on incrementally acquiring heterogeneous knowledge from both internet and publicly available datasets to reduce the tedious and expensive labelling efforts required in video annotation. An incremental transfer learning framework is presented to integrate heterogeneous source knowledge and update the annotation model incrementally during the transfer learning process. Under this framework, web images and existing action videos form the source domain to provide labelled static and motion information of the target domain videos, respectively. Moreover, according to the semantic of the source domain data, all the source domain data are partitioned into several groups. Different from traditional methods, which compare the entire target domain videos with each source group from the source domain, the authors treat the group weights as sample‐specific variables and optimise them along with new adding data. Two regularisers are used to prevent the incremental learning process from negative transfer. Experimental results on the two large‐scale consumer video datasets (i.e. multimedia event detection (MED) and Columbia consumer video (CCV)) show the effectiveness of the proposed method.