FlatStore: An Efficient Log-Structured Key-Value Storage Engine for Persistent Memory

Session: Storage--Cache is the answer, what is the question?

Authors: Youmin Chen (Tsinghua University); Youyou Lu (Tsinghua University); Fan Yang (Tsinghua University); Qing Wang (Tsinghua University); Yang Wang (The Ohio State University); Jiwu Shu (Tsinghua University)

Emerging hardware like persistent memory (PM) and high-speed NICs are promising to build efficient key-value stores. However, we observe that the small-sized access pattern in key-value stores doesn't match with the persistence granularity in PMs, leaving the PM bandwidth underutilized. This paper proposes an efficient PM-based key-value storage engine named FlatStore. Specifically, it decouples the role of a KV store into a persistent log structure for efficient storage and a volatile index for fast indexing. Upon it, FlatStore further incorporates two techniques: 1) compacted log format to maximize the batching opportunity in the log; 2) pipelined horizontal batching to steal log entries from other cores when creating a batch, thus delivering low-latency and high-throughput performance. We implement FlatStore with the volatile index of both a hash table and Masstree. We deploy FlatStore on Optane DC Persistent Memory, and our experiments show that FlatStore achieves up to 35 Mops/s with a single server node, 2.5 - 6.3 times faster than existing systems.