Premium
Learning Generative Models of 3D Structures
Author(s) -
Chaudhuri Siddhartha,
Ritchie Daniel,
Wu Jiajun,
Xu Kai,
Zhang Hao
Publication year - 2020
Publication title -
computer graphics forum
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.578
H-Index - 120
eISSN - 1467-8659
pISSN - 0167-7055
DOI - 10.1111/cgf.14020
Subject(s) - computer science , generative grammar , computer graphics , generative model , artificial intelligence , probabilistic logic , generative design , graphics , rendering (computer graphics) , human–computer interaction , computer graphics (images) , metric (unit) , operations management , economics
3D models of objects and scenes are critical to many academic disciplines and industrial applications. Of particular interest is the emerging opportunity for 3D graphics to serve artificial intelligence: computer vision systems can benefit from synthetically‐generated training data rendered from virtual 3D scenes, and robots can be trained to navigate in and interact with real‐world environments by first acquiring skills in simulated ones. One of the most promising ways to achieve this is by learning and applying generative models of 3D content: computer programs that can synthesize new 3D shapes and scenes. To allow users to edit and manipulate the synthesized 3D content to achieve their goals, the generative model should also be structure‐aware : it should express 3D shapes and scenes using abstractions that allow manipulation of their high‐level structure. This state‐of‐the‐art report surveys historical work and recent progress on learning structure‐aware generative models of 3D shapes and scenes. We present fundamental representations of 3D shape and scene geometry and structures, describe prominent methodologies including probabilistic models, deep generative models, program synthesis, and neural networks for structured data, and cover many recent methods for structure‐aware synthesis of 3D shapes and indoor scenes.