Presentation
ORBIT: Oak Ridge Base Foundation Model for Earth System Predictability
DescriptionEarth system predictability is challenged by the complexity of environmental dynamics and variables. Current AI foundation models, although advanced by large and heterogeneous data, are constrained by their size and data integration, limiting their effectiveness in addressing the full range of Earth system prediction challenges. To overcome these limitations, we introduce the Oak Ridge Base Foundation Model for Earth System Predictability (ORBIT), an advanced vision transformer model that scales up to 113 billion parameters using a novel hybrid tensor-data orthogonal parallelism technique. As the largest model of its kind, ORBIT surpasses the current climate AI foundation model size by a thousandfold. Scalability tests on the Frontier supercomputer demonstrate that ORBIT achieves 684 petaFLOPS to 1.6 exaFLOPS sustained throughput, with scaling efficiency maintained at 41% to 85% across 49,152 AMD GPUs. These breakthroughs establish new advances in AI-driven climate modeling and demonstrate promise to significantly improve Earth system predictability.
Event Type
ACM Gordon Bell Climate Modeling Finalist
TimeTuesday, 19 November 20241:30pm - 2pm EST
LocationB312-B313A
TP
Archive
view