What size should your Burst-Buffers be?

Guillaume Aupy (1) , Olivier Beaumont (2) , Lionel Eyraud-Dubois (2)

1 TADAAM - Topology-Aware System-Scale Data Management for High-Performance Computing
2 Realopt - Reformulations based algorithms for Combinatorial Optimization

Burst-Buffers are high throughput, small size intermediate storage systems typically based on SSDs or NVRAM that are designed to be used as a potential buffer between the computing nodes of a supercomputer and its main storage system consisting of hard drives. Their purpose is to absorb the bursts of I/O that many HPC applications experience (for example for saving checkpoints or data from intermediate results). In this paper, we propose a probabilistic model for evaluating the performance of Burst-Buffers. From a model of application and a data management strategy, we build a Markov chain based model of the system, that allows to quickly answer issues about dimensioning of the system: for a given set of applications, and for a given Burst-Buffer size and bandwidth, how often does the buffer overflow? We also provide extensive simulation results to validate our modeling approach.

Domaines: Calcul parallèle, distribué et partagé [cs.DC]

Guillaume Aupy, Olivier Beaumont, Lionel Eyraud-Dubois. What size should your Burst-Buffers be?. 2017. ⟨hal-01623846v1⟩