Open AccessA new strategy for curriculum learning using model distillation
Author(s) -
Kaan Karakose,
Metin Bilgin
Publication year - 2020
Publication title -
global journal of computer sciences
Language(s) - English
Resource type - Journals
ISSN - 2301-2587
DOI - 10.18844/gjcs.v10i2.5810
Subject(s) - artificial neural network , curriculum , artificial intelligence , computer science , machine learning , context (archaeology) , deep learning , sorting , set (abstract data type) , sample (material) , value (mathematics) , distillation , geography , pedagogy , psychology , chemistry , archaeology , organic chemistry , chromatography , programming language
In recent years, deep neural networks have been successful in both industry and academia, especially for computer vision tasks. Humans and animals learn much better when gradually presented in a meaningful order showing more concepts and complex samples rather than randomly presenting the information. The use of such training strategies in the context of artificial neural networks is called curriculum learning. In this study, a strategy was developed for curriculum learning. Using the CIFAR-10 and CIFAR-100 training sets, the last few layers of the pre-trained on ImageNet Xception model were trained to keep the training set knowledge in the model’s weight. Finally, a much smaller model was trained with the sample sorting methods presented using these difficulty levels. The findings obtained in this study show that the accuracy value generated when trained by the method we provided with the accuracy value trained with randomly mixed data was more than 1% for each epoch. Keywords: Curriculum learning, model distillation, deep learning, academia, neural networks.