Adaptive Auto-Partitioning in Distributed Transaction based Data Storage Models

M.Sc. defense by Vivek Shah


Data is of paramount importance in today’s world. With the advent of online services like email, blogging platforms, social networking sites, collaborative editing platforms, various scalable and efficient data storage and management systems are being designed and evaluated. Main memory based data storage systems with strong/weak/no ACID semantics are gaining importance as a research area. Almost all of these systems employ data partitioning in order to scale. However, most of these partitioning mechanisms require some form of human involvement. Significant research has been invested into designing automatic partitioning systems which can remove human involvement. The research in this area shows promising results. In this thesis, we have designed a transactional main memory based distributed data storage system with an adaptive automatic partitioning mechanism. We investigate what are the various aspects of designing such a systems and what are their trade-offs. We point out the various roadblocks in the design of the system and why they occur and how they can be bypassed. We also design and implement the adaptive and auto-partitioning mechanisms and show how various adaptive and auto- partitioning algorithms can be easily embedded in the system for evaluation. In this thesis, we show how adaptive auto-partitioning mechanisms help in making the system resilient to change in workload and lead to efficient utilization of the underlying system. 

Supervisor: Marcos Vaz Salles, DIKU

Censor: Philippe Bonnet, IT University of Copenhagen

The defense will be held in English.