Attention is all you need for boosting graph convolutional neural network
원문 초록
Graph Convolutional Neural Networks (GCNs) possess strong capabilities for processing graph data in non-grid domains. They can capture the topological logical structure and node features in graphs and integrate them into nodes' final representations. GCNs have been extensively studied in various fields, such as recommendation systems, social networks, and protein molecular structures. With the increasing application of graph neural networks, research has focused on improving their performance while compressing their size. In this work, a plug-in module named Graph Knowledge Enhancement and Distillation Module (GKEDM) is proposed. GKEDM can enhance node representations and improve the performance of GCNs by extracting and aggregating graph information via multi-head attention mechanism. Furthermore, GKEDM can serve as an auxiliary transferor for knowledge distillation. With a specially designed attention distillation method, GKEDM can distill the knowledge of large teacher models into high-performance and compact student models. Experiments on multiple datasets demonstrate that GKEDM can significantly improve the performance of various GCNs with minimal overhead. Furthermore, it can efficiently transfer distilled knowledge from large teacher networks to small student networks via attention distillation.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Light distillation for Incremental Graph Convolution Collaborative Filtering
Revisiting Graph based Social Recommendation: A Distillation Enhanced Social Graph Network
A Reproducibility Study of Product-side Fairness in Bundle Recommendation
DVFL-Net: A Lightweight Distilled Video Focal Modulation Network for Spatio-Temporal Action Recognition
NLGCL: Naturally Existing Neighbor Layers Graph Contrastive Learning for Recommendation
Boosting Parameter Efficiency in LLM-Based Recommendation through Sophisticated Pruning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.