Presentation
Scalable RPC Layer Towards Millions of IOPS per Server
DescriptionThe growing demand for large-scale AI applications brings performance challenges to parallel file systems. The performance of parallel file systems depends on both hardware components and software architecture. In parallel file systems, the performance of remote function execution is critical because most operations on a parallel file system require network communication and remote I/O. This work-in-progress paper introduces a new RPC layer that employs an optimized architecture for many-core CPUs and high-speed network devices. The RPC layer adopts a scalable task-stealing model that offers fairness in task execution and leverages the parallel performance of many-core CPUs. Our preliminary performance implementation indicates that the RPC layer can process more than four million RPC operations per second on a single server. This paper introduces the design of the RPC layer and several performance evaluation results.
