Parallel Database Processing on a 100 Node PC Cluster: Cases for Decision Support Query Processing and Data Mining

We developed a large scale PC cluster system consisting of one hundred Pentium Pro PCs interconnected by an ATM switch, and examined its performance on data warehouse processing. First, we picked up the most complex query of the standard benchmark, TPC-D, on a 100 GB database. Our PC cluster exhibited much higher performance compared with those in current benchmark reports. Second, we developed a parallel association rule mining algorithm and ran it on the PC cluster. Sufficiently high linearity was obtained. Thus we believe such commodity based PC clusters will play a very important role in large scale database processing.