Experiences and Results from a New High Performance Network and Application Monitoring Toolkit

Grid Computing capabilities are increasingly needed for scientific research. Groups such as Globus and the Particle Physics Data Grid are developing tools to meet these needs. An additional challenge is the evaluation and fine-tuning of these applications, as well as support for long term monitoring, performance analysis, and troubleshooting. In September 2001, SLAC started the development of a toolkit for studying the available bandwidth as measured by various network sensing tools and comparing that with the bandwidth achievable by various bulk data transfer applications. This study has provided experience in the challenges of deploying and using the sensor tools and transfer applications, as well as information for fine tuning the applications and analyzing their performance. The results presented in this paper include the deployment challenges, techniques for optimizing the duration of measurements, the impacts of throughput on CPU utilization, optimizing windows and parallel streams, the impact on other users, comparisons of various throughput measurement techniques, patterns of throughput behaviors, forecasting, and comparisons of active and passive measurements. We finish up with possible avenues for future development.