English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Deep generative models such as Variational Autoencoders (VAEs), GenerativeAdversarial Networks (GANs), Diffusion Models, and Transformers, have showngreat promise in a variety of applications, including image and speechsynthesis, natural language processing, and drug discovery. However, whenapplied to engineering design problems, evaluating the performance of thesemodels can be challenging, as traditional statistical metrics based onlikelihood may not fully capture the requirements of engineering applications.This paper doubles as a review and practical guide to evaluation metrics fordeep generative models (DGMs) in engineering design. We first summarize thewell-accepted `classic' evaluation metrics for deep generative models groundedin machine learning theory. Using case studies, we then highlight why thesemetrics seldom translate well to design problems but see frequent use due tothe lack of established alternatives. Next, we curate a set of design-specificmetrics which have been proposed across different research communities and canbe used for evaluating deep generative models. These metrics focus on uniquerequirements in design and engineering, such as constraint satisfaction,functional performance, novelty, and conditioning. Throughout our discussion,we apply the metrics to models trained on simple-to-visualize 2-dimensionalexample problems. Finally, we evaluate four deep generative models on a bicycleframe design problem and structural topology generation problem. In particular,we showcase the use of proposed metrics to quantify performance targetachievement, design novelty, and geometric constraints. We publicly release thecode for the datasets, models, and metrics used throughout the paper athttps://decode.mit.edu/projects/metrics/.

Beyond Statistical Similarity: Rethinking Metrics for Deep Generative Models in Engineering Design