Distributed Training of Graph Convolutional Networks using Subgraph Approximation
원문 초록
Modern machine learning techniques are successfully being adapted to data modeled as graphs. However, many real-world graphs are typically very large and do not fit in memory, often making the problem of training machine learning models on them intractable. Distributed training has been successfully employed to alleviate memory problems and speed up training in machine learning domains in which the input data is assumed to be independently identical distributed (i.i.d). However, distributing the training of non i.i.d data such as graphs that are used as training inputs in Graph Convolutional Networks (GCNs) causes accuracy problems since information is lost at the graph partitioning boundaries. In this paper, we propose a training strategy that mitigates the lost information across multiple partitions of a graph through a subgraph approximation scheme. Our proposed approach augments each sub-graph with a small amount of edge and vertex information that is approximated from all other sub-graphs. The subgraph approximation approach helps the distributed training system converge at single-machine accuracy, while keeping the memory footprint low and minimizing synchronization overhead between the machines.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
PGT-I: Scaling Spatiotemporal GNNs with Memory-Efficient Distributed Training
Demystifying Distributed Training of Graph Neural Networks for Link Prediction
DistMLIP: A Distributed Inference Platform for Machine Learning Interatomic Potentials
Hierarchical Superpixel Segmentation via Structural Information Theory
FairGP: A Scalable and Fair Graph Transformer Using Graph Partitioning
Towards the efficacy of federated prediction for epidemics on networks
토론
이 논문과 연결된 포럼 글이 아직 없습니다.