Open AccessFull body virtual try‐on with semi‐self‐supervised learning
Author(s) -
Lee HyugJae,
Koo Byumhyuk,
Ahn HaEun,
Kang Minseok,
Lee Rokkyu,
Park Gunhan
Publication year - 2021
Publication title -
electronics letters
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.375
H-Index - 146
eISSN - 1350-911X
pISSN - 0013-5194
DOI - 10.1049/ell2.12307
Subject(s) - leverage (statistics) , clothing , computer science , scheme (mathematics) , matching (statistics) , top down and bottom up design , engineering drawing , artificial intelligence , computer vision , body shape , human–computer interaction , engineering , software engineering , mathematical analysis , statistics , mathematics , archaeology , history
This paper proposes a full body virtual try‐on which handles both top and bottom garments and generates realistic try‐on images. For the full body virtual try‐on, this paper addresses lack of suitable training data to align and fit top and bottom naturally. The proposed system consists of three modules: Clothing Guide Module (CGM), Geometric Matching Module (GMM), and Try‐On Module (TOM). CGM is introduced to generate a clothing guide map (CGMap) which describes the shape of a garment on a model. Unlike the single‐garment virtual try‐on scheme, it is impractical to collect meaningful data at a large scale for the multi‐garment system. To address this problem, two novel training strategies are proposed to leverage the existing training data. First, a pseudo triplet of model‐top‐bottom is generated from a pair of model‐top or model‐bottom which are already secured. Second, the CGM network is arranged to be exposed to both top and bottom garments during training. Then, the following GMM networks warp and align the top and bottom garments. Finally, TOM synthesizes a realistic try‐on image with the aligned garment and the CGMap. Experimental results prove remarkable performance of the proposed method in the full body virtual try‐on.