Premium
image2emmet: Automatic code generation from web user interface image
Author(s) -
Xu Yong,
Bo Lili,
Sun Xiaobing,
Li Bin,
Jiang Jing,
Zhou Wei
Publication year - 2021
Publication title -
journal of software: evolution and process
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.371
H-Index - 29
eISSN - 2047-7481
pISSN - 2047-7473
DOI - 10.1002/smr.2369
Subject(s) - computer science , html , component (thermodynamics) , code (set theory) , interface (matter) , cascading style sheets , markup language , programming language , web application , task (project management) , user interface , source code , convolutional neural network , world wide web , artificial intelligence , web page , xml , operating system , physics , management , set (abstract data type) , maximum bubble pressure method , economics , thermodynamics , bubble
Web development usually follows with analyzing the functionality, designing the user interface (UI) prototype, implementing the UI by front‐end (FE) developers and implementing the REpresentational State Transfer (RESTful) application programming interface (API) by back‐end (BE) programmers. Unfortunately, web development is a tedious, cumbersome, and time‐consuming task, which makes it a challenge for the FE programmers to work in an efficient way. In this paper, we propose an approach, image2emmet, to assist FE programmers in implementing the UI. First, we collect HyperText Markup Language, Cascading Style Sheets (HTML‐CSS) dataset in an automatic and efficient way. The HTML‐CSS dataset used for model training consists of HTML‐CSS code and its display images. Second, the faster region‐based convolutional neural network (CNN) (R‐CNN) is utilized to detect the UI component. Finally, we build a model combining CNN and long short‐term memory (LSTM) to transform the UI component into the HTML‐CSS code. The empirical study demonstrates that image2emmet can achieve a precision of 80% on the UI component detection and 60% on the transformation of UI component into HTML‐CSS code.