A Metadata-Rich File System

Despite continual improvements in the performance and reliability of large scale file systems, the management of file system metadata has changed little in the past decade. The mismatch between the size and complex- ity of large scale data stores and their ability to organize and query their metadata has led to a de facto standard in which raw data is stored in traditional file systems, while related, application-specific metadata is stored in relational databases. This separation of data and meta- data requires considerable effort to maintain consistency and can result in complex, slow, and inflexible system operation. To address these problems, we have devel- oped the Quasar File System (QFS), a metadata-rich file system in which files, metadata, and file relationships are all first class objects. In contrast to hierarchical file sys- tems and relational databases, QFS defines a graph data model composed of files and their relationships. QFS in- cludes Quasar, an XPATH-extended query language for searching the file system. Results from our QFS pro- totype show the effectiveness of this approach. Com- pared to the defacto standard, the QFS prototype shows superior ingest performance and comparable query per- formance on user metadata-intensive operations and su- perior performance on normal file metadata operations.