Presentation
High-Performance Object Storage: I/O for the Exascale Era
DescriptionRecently we have seen a change in the diversity of applications utilizing high-performance computing (HPC) from primarily computational simulation approaches, to a more varied application mix including machine learning and data analytics. With this diversification in workloads, there has also been a diversification in I/O patterns; the movements in, and requirements on, data storage and access. Data storage technologies in HPC have long been optimized for large scale bulk operations focused on high-bandwidth with relatively low volumes of metadata operations. However, many applications now exhibit non-optimal I/O patterns for large scale parallel filesystems, with large amounts of small I/O operations, non-contiguous data access, and increases in read as well as write I/O loads.
Parallel filesystems, such as Lustre and Storage Scale, have been optimized and extended to provided higher metadata performance and to better handle small I/O operations. However, the underlying approach of POSIX-like I/O, with block-sized read and write operations, and file-level data storage, set limitations on the overall performance and functionality that such approaches can achieve. This tutorial will educate attendees in the design and usage of object stores, as alternatives to filesystems, using Ceph and DAOS as examples, through hands-on exercise and lecture sessions.
Parallel filesystems, such as Lustre and Storage Scale, have been optimized and extended to provided higher metadata performance and to better handle small I/O operations. However, the underlying approach of POSIX-like I/O, with block-sized read and write operations, and file-level data storage, set limitations on the overall performance and functionality that such approaches can achieve. This tutorial will educate attendees in the design and usage of object stores, as alternatives to filesystems, using Ceph and DAOS as examples, through hands-on exercise and lecture sessions.
Event Type
Tutorial
TimeSunday, 17 November 20241:30pm - 5pm EST
LocationB213
Architecture
Emerging Technologies
I/O, Storage, Archive
Scalable Data Mining
TUT