Presentation
SIGN IN TO VIEW THIS PRESENTATION Sign In
Jarvis: Towards a Shared, User-Friendly, and Reproducible I/O Infrastructure
DescriptionHardware is becoming increasingly heterogeneous in modern high-performance computing clusters. However, computing environments for developing tools to harness these technologies are not easily available to researchers. This work showcases the need for a new high-pace, heterogeneous I/O research cluster and presents a novel software deployment framework named Jarvis to manage its hardware diversity. Jarvis is an extensible Python framework that allows users to create packages that deploy, manage, and monitor software, including complex applications (e.g., scientific simulations), support tools (e.g., Darshan, GDB), and storage systems (e.g., Lustre, DAOS). These packages can be combined to form complex deployment pipelines. To ensure pipelines are portable across hardware, Jarvis defines a novel resource graph schema file, which is a snapshot of a cluster's machine-specific information. This schema can be queried by Jarvis packages to deploy software across diverse hardware compositions with minimal user effort.
Event Type
Workshop
TimeSunday, 17 November 20244pm - 4:05pm EST
LocationB309
Data Movement and Memory
I/O, Storage, Archive
W