Presentation
Containerizing Real-time Streaming of Scientific Data in HPC: Challenges and Solutions
DescriptionIn this paper, we will explore the containerization focusing on scientific simulation data such as NWCHEM and the Open Catalyst OCP. Additionally, how singularity containers are used to containerize HPC applications. We run different machine learning frameworks with and without Singularity containers show their performance and challenges of integrating containerization technologies within High-Performance Computing (HPC) environments, particularly for applications in computational chemistry and materials science. We also highlighted the limitations of traditional containerization tools such as Docker in supercomputing contexts and introduces Singularity as a more suitable alternative tailored to HPC requirements. By analyzing the containerization process of NWChem and OCP project at the PERLMUTTER supercomputer at LBNL, we illustrate how Singularity effectively addresses HPC-specific needs while ensuring performance and scalability. Our findings offer valuable insights for optimizing containerization strategies for complex scientific software in advanced computing environments.
Author/Presenter
