Presentation
Designing Efficient Data Reduction Approaches for Multi-Resolution Simulations on HPC Systems
DescriptionAs supercomputers advance towards exascale capabilities, computational intensity increases significantly, and the volume of data requiring storage and transmission experiences exponential growth. Multi-resolution methods, such as Adaptive Mesh Refinement (AMR), have emerged as an effective solution to address these challenges. Concurrently, error-bounded lossy compression is recognized as one of the most efficient approaches to tackle the latter issue. Despite their respective advantages, few attempts have been made to investigate how the multi-resolution method and error-bounded lossy compression can function together.
To address this gap, this dissertation introduces a series of optimizations for data reduction solutions in multi-resolution simulations:
(1) This dissertation first enhances the offline compression quality of multi-resolution data for different state-of-the-art scientific compressors by adaptively preprocessing the data and optimizing the compressor.
(2) This dissertation then presents a novel in-situ lossy compression framework, utilizing HDF5 and enhanced SZ2, specifically tailored for real-world AMR applications. This framework can reduce I/O costs and improve compression quality.
(3) Furthermore, to extend the usability of multi-resolution techniques, this dissertation introduces a workflow for multi-resolution data compression, applicable to both uniform and AMR simulations. Initially, the workflow employs a Region of Interest (ROI) extraction approach to enable multi-resolution methods for uniform data. Subsequently, to bridge the gap between multi-resolution techniques and lossy compressors, we optimize three distinct compressors, ensuring their optimal performance on multi-resolution data. Lastly, we incorporate an advanced uncertainty visualization method into our workflow to help users understand the potential impacts of lossy compression.
To address this gap, this dissertation introduces a series of optimizations for data reduction solutions in multi-resolution simulations:
(1) This dissertation first enhances the offline compression quality of multi-resolution data for different state-of-the-art scientific compressors by adaptively preprocessing the data and optimizing the compressor.
(2) This dissertation then presents a novel in-situ lossy compression framework, utilizing HDF5 and enhanced SZ2, specifically tailored for real-world AMR applications. This framework can reduce I/O costs and improve compression quality.
(3) Furthermore, to extend the usability of multi-resolution techniques, this dissertation introduces a workflow for multi-resolution data compression, applicable to both uniform and AMR simulations. Initially, the workflow employs a Region of Interest (ROI) extraction approach to enable multi-resolution methods for uniform data. Subsequently, to bridge the gap between multi-resolution techniques and lossy compressors, we optimize three distinct compressors, ensuring their optimal performance on multi-resolution data. Lastly, we incorporate an advanced uncertainty visualization method into our workflow to help users understand the potential impacts of lossy compression.