Presentation
SiPearl’s Rhea processor: high-bandwidth memory, ultimate solution for Large Language Models
DescriptionUntil now, most LLM are managed using GPU or dedicated accelerators. But, their cost combined to their low availability on the market and their level of energy consumption are prompting us to turn to other solutions. In this context, SiPearl’s high-performance energy-efficient processor with built-in High Bandwidth Memory (HBM), Rhea, will be the ultimate solution for LLM workloads. The LLM’s workflow can be divided into three steps: 1) sanitizing and extracting features, 2) building/training founding models, and 3) fine-tuning and using models. While collecting, identifying and extracting relevant features from the raw data (1st step) is already done on CPU, the other steps are still performed on GPU.
This talk describes why and how other tasks can be carried out more advantageously on SiPearl’s processor with built-in HBM. It covers inference, fine-tuning and training and demonstrates among other things the resilience of Rhea which is more flexible to model changes than solutions currently in use.
This talk describes why and how other tasks can be carried out more advantageously on SiPearl’s processor with built-in HBM. It covers inference, fine-tuning and training and demonstrates among other things the resilience of Rhea which is more flexible to model changes than solutions currently in use.
Presenter