Data partitioning and load balancing in parallel disk systems | Zendy

Peter Scheuermann | Zendy; Gerhard Weikum | Zendy; Peter Zabback | Zendy

AI Assistant Blog Pricing

Home ZAIA Blog

Open Access

Data partitioning and load balancing in parallel disk systems

Author(s) -

Peter Scheuermann,

Gerhard Weikum,

Peter Zabback

Publication year - 1998

Publication title -

the vldb journal

Language(s) - English

Resource type - Journals

SCImago Journal Rank - 0.653

H-Index - 90

eISSN - 0949-877X

pISSN - 1066-8888

DOI - 10.1007/s007780050053

Subject(s) - computer science , data striping , heuristics , load balancing (electrical power) , overhead (engineering) , parallelism (grammar) , distributed computing , throughput , parallel computing , simple (philosophy) , file system , response time , operating system , philosophy , epistemology , wireless , geometry , mathematics , grid

Parallel disk systems provide opportunities for exploiting I/O parallelism in two possible ways, namely via inter-request and intra-request parallelism. In this paper, we discuss the main issues in performance tuning of such systems, namely striping and load balancing, and show their relationship to response time and throughput. We outline the main components of an intelligent, self-reliant file system that aims to optimize striping by taking into account the requirements of the applications, and performs load balancing by judicious file allocation and dynamic redistributions of the data when access patterns change. Our system uses simple but effective heuristics that incur only little overhead. We present performance experiments based on synthetic workloads and real-life traces.

The content you want is available to Zendy users.

Already have an account? Click here to sign in.

Having issues? You can contact us here

Accelerating Research