Presentation
Understanding and Predicting Cross-Application I/O Interference in HPC Storage Systems
DescriptionOn high performance computing systems, multiple concurrent workloads may read and write vast amounts of data stored through shared storage servers, hence competition for I/O resources between workloads is inevitable. Previous work has thoroughly recognized the impact of such competition-introduced resource contention, highlighting its potential to impact the performance of individual applications significantly. However, no prior work on such an issue has investigated the quantitative impact of inter-application I/O contention on individual applications. In this work, we first exemplify the dynamics of I/O interference towards I/O patterns and system status. We then propose a framework for collecting fine-grained I/O traces from applications and concurrent server-side metrics and train a neural network to accurately predict the existence of I/O interference and its potential impacts. Our results show that our model can accurately predict the impact of I/O interference with F1 scores exceeding 90% for both synthetic benchmarks and real-world applications.
