Data Management Systems Seminar: FoundationDB at Snowflake: Architecture and Internals

Title

FoundationDB at Snowflake: Architecture and Internals

Abstract

FoundationDB is a distributed key value store with support for strictly serializable multi-row transactions. It runs on thousands of machines at multiple companies, including Snowflake, Apple, VMWare, and Epic Games. This product was initially developed by the FoundationDB company and later acquired and made open source by Apple. Today Snowflake is a major contributor to the open source project and a FoundationDB cluster runs at the heart of every Snowflake deployment.

This talk will be divided into three sections. First, I will give a very brief overview of how Snowflake works and why FoundationDB is such an essential part of it. Then I will describe the architecture and the inner workings of FoundationDB. In this part I will also address  the current challenges and the exciting projects we’re working on. Lastly, I will describe FoundationDB’s unique and comprehensive testing strategy, which is the secret for its success.

BioPortrait of Markus Pilman

Markus Pilman joined Snowflake in April 2016 as a software engineer where he works on FoundationDB as a tech lead. The FoundationDB team which is part of the persistent metadata team. Snowflake and Apple are currently the largest contributors to the FoundationDB open source project. Markus is currently working on projects to improve scalability and latencies. Before joining Snowflake, he acquired a Dr. sc. ETH Zurich in Switzerland where he worked under the supervision of Donald Kossmann on distributed key value stores. In his free time he enjoys hiking, camping, and photography.

Contact Person

Yongluan Zhou