NeutronOrch: Rethinking Sample-based GNN Training under CPU-GPU Heterogeneous Environments
원문 초록
Graph Neural Networks (GNNs) have demonstrated outstanding performance in various applications. Existing frameworks utilize CPU-GPU heterogeneous environments to train GNN models and integrate mini-batch and sampling techniques to overcome the GPU memory limitation. In CPU-GPU heterogeneous environments, we can divide sample-based GNN training into three steps: sample, gather, and train. Existing GNN systems use different task orchestrating methods to employ each step on CPU or GPU. After extensive experiments and analysis, we find that existing task orchestrating methods fail to fully utilize the heterogeneous resources, limited by inefficient CPU processing or GPU resource contention. In this paper, we propose NeutronOrch, a system for sample-based GNN training that incorporates a layer-based task orchestrating method and ensures balanced utilization of the CPU and GPU. NeutronOrch decouples the training process by layer and pushes down the training task of the bottom layer to the CPU. This significantly reduces the computational load and memory footprint of GPU training. To avoid inefficient CPU processing, NeutronOrch only offloads the training of frequently accessed vertices to the CPU and lets GPU reuse their embeddings with bounded staleness. Furthermore, NeutronOrch provides a fine-grained pipeline design for the layer-based task orchestrating method, fully overlapping different tasks on heterogeneous resources while strictly guaranteeing bounded staleness. The experimental results show that compared with the state-of-the-art GNN systems, NeutronOrch can achieve up to 11.51x performance speedup.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
MathOptAI.jl: Embed trained machine learning predictors into JuMP models
ConsumerBench: Benchmarking Generative AI Applications on End-User Devices
Versatile and Fast Location-Based Private Information Retrieval with Fully Homomorphic Encryption over the Torus
SecONNds: Secure Outsourced Neural Network Inference on ImageNet
Plug-and-Play Linear Attention for Pre-trained Image and Video Restoration Models
FlashDMoE: Fast Distributed MoE in a Single Kernel
토론
이 논문과 연결된 포럼 글이 아직 없습니다.