Exploring regularities in software with statistical models and their applications

Software systems are becoming popular. They are used with di erent platforms for di erent applications. Software systems are developed with support from programming languages, which help developers work conveniently. Programming languages can have di erent paradigms with di erent form, syntactic structures, keywords, representation ways. In many cases, however, programming languages are similar in di erent important aspects: 1. They are used to support description of speci c tasks, 2. Source codes are written in languages and includes a limit set of distinctive tokens, many tokens are repeated like keywords, function calls, and 3. They follow speci c syntactic rules to make machine understanding. Those points also re ect the similarity between programming language and natural language. Due to its critical role in many applications, natural language processing (NLP) has been studied much and given many promising results like automatic cross-language translation, speech-to-text, information searching, etc. It is interesting to observe if there are similar characteristics between natural language and programming language and whether techniques in NLP can be reused for programming language processing? Recent works in software engineering (SE) shows that their similarities between NLP and programming language processing and techniques in NLP can be reused for PLP. This dissertation introduces my works with contributions in study of characteristics of programming languages, the models which employed them and the main applications that show the usefulness of the proposed models. Study in both three aspects has draw interests from software engineering community and received awards due to their innovation and applicability I hope that this dissertation will bring a systematic view of how advantage techniques in natural language processing and machine learning can be re-used and give huge bene t for programming language processing, and how those techniques are adapted with characteristics of programming language and software systems.