Close

Presentation

Scalable Multi-Facility Workflows for Artificial Intelligence Applications in Climate Research
DescriptionEarth observation and earth system models are sources of vast, multi-modal datasets that are invaluable for advancing climate and environmental research. However, their scale and complexity pose challenges for processing and analysis. In this paper we discuss our experiences in developing a scientific research application using an automated multi-facility workflow that orchestrates data collection, preprocessing, artificial intelligence (AI) inferencing, and data movement across diverse computational resources, leveraging the Advanced Computing Ecosystem Testbed at the Oak Ridge Leadership Computing Facility (OLCF). We demonstrate that our AI application workflow can be seamlessly integrated and orchestrated across research facilities to extract new scientific insights from climate datasets using data intensive computational methods. The results indicate that the multi-facility workflow reduces processing time, enhances scalability, and maintains high efficiency across varying workloads. Our workflow processes 12,000 high-resolution satellite images in 44 seconds using 80 workers distributed across 10 nodes on the OLCF systems.