Premium
Sketch‐based modeling with a differentiable renderer
Author(s) -
Xiang Nan,
Wang Ruibin,
Jiang Tao,
Wang Li,
Li Yanran,
Yang Xiaosong,
Zhang Jianjun
Publication year - 2020
Publication title -
computer animation and virtual worlds
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.225
H-Index - 49
eISSN - 1546-427X
pISSN - 1546-4261
DOI - 10.1002/cav.1939
Subject(s) - sketch , computer science , rendering (computer graphics) , sketch recognition , artificial intelligence , differentiable function , ambiguity , pipeline (software) , image based modeling and rendering , line drawings , computer graphics (images) , computer vision , algorithm , engineering drawing , gesture recognition , mathematical analysis , gesture , mathematics , programming language , engineering
Sketch‐based modeling aims to recover three‐dimensional (3D) shape from two‐dimensional line drawings. However, due to the sparsity and ambiguity of the sketch, it is extremely challenging for computers to interpret line drawings of physical objects. Most conventional systems are restricted to specific scenarios such as recovering for specific shapes, which are not conducive to generalize. Recent progress of deep learning methods have sparked new ideas for solving computer vision and pattern recognition issues. In this work, we present an end‐to‐end learning framework to predict 3D shape from line drawings. Our approach is based on a two‐steps strategy, it converts the sketch image to its normal image, then recover the 3D shape subsequently. A differentiable renderer is proposed and incorporated into this framework, it allows the integration of the rendering pipeline with neural networks. Experimental results show our method outperforms the state‐of‐art, which demonstrates that our framework is able to cope with the challenges in single sketch‐based 3D shape modeling.