Open AccessKnowledge Support and Automation for Performance Analysis with PerfExplorer 2.0
Author(s) -
Kevin Huck,
Allen D. Malony,
Sameer Shende,
Alan Morris
Publication year - 2008
Publication title -
scientific programming
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.269
H-Index - 36
eISSN - 1875-919X
pISSN - 1058-9244
DOI - 10.1155/2008/985194
Subject(s) - computer science , scalability , scripting language , automation , metadata , python (programming language) , cluster analysis , data mining , process (computing) , exploratory data analysis , software engineering , data science , machine learning , database , programming language , world wide web , mechanical engineering , engineering
The integration of scalable performance analysis in parallel development tools is difficult. The potential size of data sets and the need to compare results from multiple experiments presents a challenge to manage and process the information. Simply to characterize the performance of parallel applications running on potentially hundreds of thousands of processor cores requires new scalable analysis techniques. Furthermore, many exploratory analysis processes are repeatable and could be automated, but are now implemented as manual procedures. In this paper, we will discuss the current version of PerfExplorer, a performance analysis framework which provides dimension reduction, clustering and correlation analysis of individual trails of large dimensions, and can perform relative performance analysis between multiple application executions. PerfExplorer analysis processes can be captured in the form of Python scripts, automating what would otherwise be time-consuming tasks. We will give examples of large-scale analysis results, and discuss the future development of the framework, including the encoding and processing of expert performance rules, and the increasing use of performance metadata.