PyTorchKR Papers

최신 논문

총 1,299편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:1811.00210공식 코드 · Papers with Code

Online Planner Selection with Graph Neural Networks and Adaptive Scheduling

Automated planning is one of the foundational areas of AI. Since no single planner can work well for all tasks and domains, portfolio-based techniques have become increasingly popular in recent years. In particular, deep learning emerges as a promising methodology for online planner selection. Owing to the recent development of structural graph representations of planning tasks, we propose a graph neural network (GNN) approach to selecting candidate planners. GNNs are advantageous over a straightforward alternative, the convolutional neural networks, in that they are invariant to node permutations and that they incorporate node labels for better inference. Additionally, for cost-optimal planning, we propose a two-stage adaptive scheduling method to further improve the likelihood that a given task is solved in time. The scheduler may switch at halftime to a different planner, conditioned on the observed performance of the first one. Experimental results validate the effectiveness of the proposed method against strong baselines, both deep learning and non-deep learning based. The code is available at \url{https://github.com/matenure/GNN_planner}.

arXiv:1810.10627공식 코드 · Papers with Code

Streaming Graph Neural Networks

Graphs are essential representations of many real-world data such as social networks. Recent years have witnessed the increasing efforts made to extend the neural network models to graph-structured data. These methods, which are usually known as the graph neural networks, have been applied to advance many graphs related tasks such as reasoning dynamics of the physical system, graph classification, and node classification. Most of the existing graph neural network models have been designed for static graphs, while many real-world graphs are inherently dynamic. For example, social networks are naturally evolving as new users joining and new relations being created. Current graph neural network models cannot utilize the dynamic information in dynamic graphs. However, the dynamic information has been proven to enhance the performance of many graph analytic tasks such as community detection and link prediction. Hence, it is necessary to design dedicated graph neural networks for dynamic graphs. In this paper, we propose DGNN, a new {\bf D}ynamic {\bf G}raph {\bf N}eural {\bf N}etwork model, which can model the dynamic information as the graph evolving. In particular, the proposed framework can keep updating node information by capturing the sequential information of edges (interactions), the time intervals between edges and information propagation coherently. Experimental results on various dynamic graphs demonstrate the effectiveness of the proposed framework.

arXiv:1810.08305공식 코드 · Papers with Code

Open Vocabulary Learning on Source Code with a Graph-Structured Cache

Machine learning models that take computer program source code as input typically use Natural Language Processing (NLP) techniques. However, a major challenge is that code is written using an open, rapidly changing vocabulary due to, e.g., the coinage of new variable and method names. Reasoning over such a vocabulary is not something for which most NLP methods are designed. We introduce a Graph-Structured Cache to address this problem; this cache contains a node for each new word the model encounters with edges connecting each word to its occurrences in the code. We find that combining this graph-structured cache strategy with recent Graph-Neural-Network-based models for supervised learning on code improves the models' performance on a code completion task and a variable naming task --- with over $100\%$ relative improvement on the latter --- at the cost of a moderate increase in computation time.

arXiv:1809.05241공식 코드 · Papers with Code

Graph Pattern Mining and Learning through User-defined Relations (Extended Version)

In this work we propose R-GPM, a parallel computing framework for graph pattern mining (GPM) through a user-defined subgraph relation. More specifically, we enable the computation of statistics of patterns through their subgraph classes, generalizing traditional GPM methods. R-GPM provides efficient estimators for these statistics by employing a MCMC sampling algorithm combined with several optimizations. We provide both theoretical guarantees and empirical evaluations of our estimators in application scenarios such as stochastic optimization of deep high-order graph neural network models and pattern (motif) counting. We also propose and evaluate optimizations that enable improvements of our estimators accuracy, while reducing their computational costs in up to 3-orders-of-magnitude. Finally,we show that R-GPM is scalable, providing near-linear speedups on 44 cores in all of our tests.

arXiv:1809.02840공식 코드 · Papers with Code

Neural Guided Constraint Logic Programming for Program Synthesis

Synthesizing programs using example input/outputs is a classic problem in artificial intelligence. We present a method for solving Programming By Example (PBE) problems by using a neural model to guide the search of a constraint logic programming system called miniKanren. Crucially, the neural model uses miniKanren's internal representation as input; miniKanren represents a PBE problem as recursive constraints imposed by the provided examples. We explore Recurrent Neural Network and Graph Neural Network models. We contribute a modified miniKanren, drivable by an external agent, available at https://github.com/xuexue/neuralkanren. We show that our neural-guided approach using constraints can synthesize programs faster in many cases, and importantly, can generalize to larger problems.

arXiv:1807.05560공식 코드 · Papers with Code

DeepInf: Social Influence Prediction with Deep Learning

Social and information networking activities such as on Facebook, Twitter, WeChat, and Weibo have become an indispensable part of our everyday life, where we can easily access friends' behaviors and are in turn influenced by them. Consequently, an effective social influence prediction for each user is critical for a variety of applications such as online recommendation and advertising. Conventional social influence prediction approaches typically design various hand-crafted rules to extract user- and network-specific features. However, their effectiveness heavily relies on the knowledge of domain experts. As a result, it is usually difficult to generalize them into different domains. Inspired by the recent success of deep neural networks in a wide range of computing applications, we design an end-to-end framework, DeepInf, to learn users' latent feature representation for predicting social influence. In general, DeepInf takes a user's local network as the input to a graph neural network for learning her latent social representation. We design strategies to incorporate both network structures and user-specific features into convolutional neural and attention networks. Extensive experiments on Open Academic Graph, Twitter, Weibo, and Digg, representing different types of social and information networks, demonstrate that the proposed end-to-end model, DeepInf, significantly outperforms traditional feature engineering-based approaches, suggesting the effectiveness of representation learning for social applications.

arXiv:1806.09206공식 코드 · Papers with Code

N-Gram Graph: Simple Unsupervised Representation for Graphs, with Applications to Molecules

Machine learning techniques have recently been adopted in various applications in medicine, biology, chemistry, and material engineering. An important task is to predict the properties of molecules, which serves as the main subroutine in many downstream applications such as virtual screening and drug design. Despite the increasing interest, the key challenge is to construct proper representations of molecules for learning algorithms. This paper introduces the N-gram graph, a simple unsupervised representation for molecules. The method first embeds the vertices in the molecule graph. It then constructs a compact representation for the graph by assembling the vertex embeddings in short walks in the graph, which we show is equivalent to a simple graph neural network that needs no training. The representations can thus be efficiently computed and then used with supervised learning methods for prediction. Experiments on 60 tasks from 10 benchmark datasets demonstrate its advantages over both popular graph neural networks and traditional representation methods. This is complemented by theoretical analysis showing its strong representation and prediction power.

arXiv:1706.05674공식 코드 · Papers with Code

Knowledge Transfer for Out-of-Knowledge-Base Entities: A Graph Neural Network Approach

Knowledge base completion (KBC) aims to predict missing information in a knowledge base.In this paper, we address the out-of-knowledge-base (OOKB) entity problem in KBC:how to answer queries concerning test entities not observed at training time. Existing embedding-based KBC models assume that all test entities are available at training time, making it unclear how to obtain embeddings for new entities without costly retraining. To solve the OOKB entity problem without retraining, we use graph neural networks (Graph-NNs) to compute the embeddings of OOKB entities, exploiting the limited auxiliary knowledge provided at test time.The experimental results show the effectiveness of our proposed model in the OOKB setting.Additionally, in the standard KBC setting in which OOKB entities are not involved, our model achieves state-of-the-art performance on the WordNet dataset. The code and dataset are available at https://github.com/takuo-h/GNN-for-OOKB

arXiv:1711.04043공식 코드 · Papers with Code

Few-Shot Learning with Graph Neural Networks

We propose to study the problem of few-shot learning with the prism of inference on a partially observed graphical model, constructed from a collection of input images whose label can be either observed or not. By assimilating generic message-passing inference algorithms with their neural-network counterparts, we define a graph neural network architecture that generalizes several of the recently proposed few-shot learning models. Besides providing improved numerical performance, our framework is easily extended to variants of few-shot learning, such as semi-supervised or active learning, demonstrating the ability of graph-based models to operate well on 'relational' tasks.

arXiv:1802.09691공식 코드 · Papers with Code

Link Prediction Based on Graph Neural Networks

Link prediction is a key problem for network-structured data. Link prediction heuristics use some score functions, such as common neighbors and Katz index, to measure the likelihood of links. They have obtained wide practical uses due to their simplicity, interpretability, and for some of them, scalability. However, every heuristic has a strong assumption on when two nodes are likely to link, which limits their effectiveness on networks where these assumptions fail. In this regard, a more reasonable way should be learning a suitable heuristic from a given network instead of using predefined ones. By extracting a local subgraph around each target link, we aim to learn a function mapping the subgraph patterns to link existence, thus automatically learning a `heuristic' that suits the current network. In this paper, we study this heuristic learning paradigm for link prediction. First, we develop a novel $\gamma$-decaying heuristic theory. The theory unifies a wide range of heuristics in a single framework, and proves that all these heuristics can be well approximated from local subgraphs. Our results show that local subgraphs reserve rich information related to link existence. Second, based on the $\gamma$-decaying theory, we propose a new algorithm to learn heuristics from local subgraphs using a graph neural network (GNN). Its experimental results show unprecedented performance, working consistently well on a wide range of problems.

arXiv:1805.05081공식 코드 · Papers with Code

Constructing Narrative Event Evolutionary Graph for Script Event Prediction

Script event prediction requires a model to predict the subsequent event given an existing event context. Previous models based on event pairs or event chains cannot make full use of dense event connections, which may limit their capability of event prediction. To remedy this, we propose constructing an event graph to better utilize the event network information for script event prediction. In particular, we first extract narrative event chains from large quantities of news corpus, and then construct a narrative event evolutionary graph (NEEG) based on the extracted chains. NEEG can be seen as a knowledge base that describes event evolutionary principles and patterns. To solve the inference problem on NEEG, we present a scaled graph neural network (SGNN) to model event interactions and learn better event representations. Instead of computing the representations on the whole graph, SGNN processes only the concerned nodes each time, which makes our model feasible to large-scale graphs. By comparing the similarity between input context event representations and candidate event representations, we can choose the most reasonable subsequent event. Experimental results on widely used New York Times corpus demonstrate that our model significantly outperforms state-of-the-art baseline methods, by using standard multiple choice narrative cloze evaluation.

arXiv:1705.08415공식 코드 · Papers with Code

Supervised Community Detection with Line Graph Neural Networks

Traditionally, community detection in graphs can be solved using spectral methods or posterior inference under probabilistic graphical models. Focusing on random graph families such as the stochastic block model, recent research has unified both approaches and identified both statistical and computational detection thresholds in terms of the signal-to-noise ratio. By recasting community detection as a node-wise classification problem on graphs, we can also study it from a learning perspective. We present a novel family of Graph Neural Networks (GNNs) for solving community detection problems in a supervised learning setting. We show that, in a data-driven manner and without access to the underlying generative models, they can match or even surpass the performance of the belief propagation algorithm on binary and multi-class stochastic block models, which is believed to reach the computational threshold. In particular, we propose to augment GNNs with the non-backtracking operator defined on the line graph of edge adjacencies. Our models also achieve good performance on real-world datasets. In addition, we perform the first analysis of the optimization landscape of training linear GNNs for community detection problems, demonstrating that under certain simplifications and assumptions, the loss values at local and global minima are not far apart.

arXiv:2312.11018v4cs.IR공식 코드 · Papers with Code

번들 추천을 위한 초그래프 강화 듀얼 합성곱 신경망

Hypergraph-Enhanced Dual Convolutional Network for Bundle Recommendation

번들 추천(Bundle recommendation)은 개별 아이템이 아닌 연관된 아이템들의 묶음을 순위화합니다. 번들 추천의 핵심 과제는 번들의 순위를 매기는 데 필요한 신호를 잃지 않으면서 사용자 선호도, 아이템 상호작용 및 번들 구성을 연결하는 것입니다. 본 연구에서는 사용자-번일, 사용자-아이템, 번들-아이템 상호작용과 사용자 내부 및 번들 내부 관계를 포함하는 완전 초그래프(complete hypergraph)를 구축하는 초그래프 강화 듀얼 합성곱 신경망(Hypergraph-Enhanced Dual Convolutional Neural Network, HED)을 제안합니다. HED는 완전 초그래프 전파를 사용자-번들 브랜치와 결합하여, 추천에 특화된 신호를 보존하는 동시에 아이템을 인식하는 고차 컨텍스트가 순위 산정에 반영되도록 합니다. NetEase 데이터셋에서 HED-128은 보고된 6개 지표 전반에 걸쳐 가장 강력한 기준선(baseline) 대비 5.04~6.97% 성능 향상을 달성했으며, Youshu에서는 HED-64가 1.87~4.56% 향상되었습니다. 절제 연구(ablation) 결과는 사용자-번들 브랜치와 유형 내부(intra-type) 관계 모두의 기여도를 뒷받침하며, 민감도 분석을 통해 주요 하이퍼파라미터의 안정적인 작동 범위를 확인했습니다. 또한 메모리 비용을 포함하여 완전 초그래프의 연산 트레이드오프를 정량화했습니다. 이러한 실증적 근거는 평가된 두 가지 번들 추천 데이터셋에서 HED의 우수성을 입증하는 동시에 자원 한계를 명확히 보여줍니다. 코드와 데이터셋은 논문 게재 시 공개될 예정입니다.

arXiv:2110.14961v4cs.LG공식 코드 · Papers with Code

상호작용하는 동적 시스템을 위한 회전-변위 로컬 좌표계

Roto-translated Local Coordinate Frames For Interacting Dynamical Systems

상호작용 모델링은 고도로 비선형적이며 시간에 따라 변하는 동작을 보이는 상호작용 객체 시스템, 즉 복잡한 동적 시스템(dynamical systems)을 학습하는 데 매우 중요합니다. 이러한 시스템의 상당 부분은 기하학적 그래프($\textit{geometric graphs}$), 즉 교통 상황의 차량과 같이 $\textit{임의로}$ 선택된 글로벌 좌표계 하에서 유클리드 공간에 위치한 노드(node)를 갖는 그래프로 정형화될 수 있습니다. 임의의 글로벌 좌표계에도 불구하고, 해당 동적 시스템을 지배하는 동역학은 회전 및 변위(translation)에 대해 불변이며, 이는 갈릴레이 불변성($\textit{Galilean invariance}$)으로도 알려져 있습니다. 이러한 불변성을 무시하면 일반화 성능이 저하되므로, 본 연구에서는 상호작용 동적 시스템의 기하학적 그래프에 회전-변위 불변성을 유도하기 위해 노드 객체별 로컬 좌표계(local coordinate frames)를 제안합니다. 또한, 로컬 좌표계를 통해 그래프 신경망(GNN)에서 비등방성 필터링(anisotropic filtering)을 자연스럽게 정의할 수 있습니다. 교통 상황, 3D 모션 캡처, 충돌 입자에 대한 실험을 통해 제안된 접근 방식이 최근 SOTA(state-of-the-art)를 여유 있게 능가함을 입증합니다.

arXiv:2501.06002v4cs.LG공식 코드 · Papers with Code

DeltaGNN: 정보 흐름 제어 기능을 갖춘 그래프 신경망

DeltaGNN: Graph Neural Network with Information Flow Control

그래프 신경망(GNN)은 메시지 전달(message passing) 과정에서 재귀적인 이웃 집계를 통해 그래프 구조화된 데이터를 처리하도록 설계된 대표적인 심층 학습(deep learning) 모델입니다. 준지도 노드 분류(semi-supervised node classification)에 적용될 때, 메시지 전달을 통해 GNN은 단거리 공간 상호작용을 파악할 수 있지만, 동시에 지나친 평활화(over-smoothing) 및 지나친 압축(over-squashing) 문제를 겪게 됩니다. 이러한 문제들은 모델 표현력을 저해하며, 그래프 내의 장거리 노드 상호작용(LRI)을 포착하기 위해 더 깊은 모델을 사용하는 것을 가로막습니다. LRI 탐지를 위한 기존의 주요 해결책들은 높은 시간 복잡도로 인해 대규모 그래프를 처리하기에 비용이 너무 많이 들거나, 다양한 그래프 구조 전반에 걸쳐 일반화되지 못합니다. 이러한 한계를 해결하기 위해, 본 연구에서는 이론적 근거를 바탕으로 선형적인 계산 오버헤드만으로 지나친 평활화와 지나친 압축 문제를 해결하는 새로운 연결성 척도인 \emph{정보 흐름 점수(information flow score)}를 활용하는 \emph{정보 흐름 제어(information flow control)} 메커니즘을 제안합니다. 이 메커니즘을 기반으로, 본 논문에서는 우리가 아는 한 장거리 및 단거리 상호작용 탐지를 위해 선형 계산 및 메모리 복잡도 오버헤드를 갖추어 \textit{확장 가능(scalable)}하면서도, 다양한 동질성(homophily), 밀도, 토폴로지를 갖는 그래프를 효과적으로 처리할 수 있어 \textit{일반화 가능(generalizable)}한 최초의 아키텍처 중 하나인 DeltaGNN을 소개합니다. 다양한 크기, 토폴로지, 밀도, 동질성 비율을 가진 그래프를 포함한 10개의 실제 데이터셋에서 모델을 벤치마크 평가하였으며, 제한된 계산 복잡도로 우수한 성능을 입증했습니다. 제안된 방법의 구현체는 https://github.com/basiralab/DeltaGNN 에서 공개되어 이용할 수 있습니다.

arXiv:2209.00546v5stat.ML공식 코드 · Papers with Code

MSGNN: 새로운 자기 부호 라플라시안 기반의 스펙트럼 그래프 신경망

MSGNN: A Spectral Graph Neural Network Based on a Novel Magnetic Signed Laplacian

부호가 있고 방향성이 있는 네트워크는 실제 응용 분야 도처에 존재합니다. 그러나 이러한 네트워크를 위한 스펙트럼 그래프 신경망(GNN)을 제안한 연구는 비교적 적었습니다. 본 논문에서는 부호 그래프에서의 부호 라플라시안과 방향성 그래프에서의 자기 라플라시안(magnetic Laplacian)을 자연스럽게 일반화한, 자기 부호 라플라시안(magnetic signed Laplacian)이라 부르는 부호 방향 라플라시안 행렬을 도입합니다. 이어 이 행렬을 사용하여 새로운 효율적인 스펙트럼 GNN 아키텍처를 구축하고, 노드(node) 클러스터링 및 링크 예측 태스크 모두에서 광범위한 실험을 수행합니다. 이 실험들에서는 부호 정보 관련 태스크, 방향 정보 관련 태스크, 그리고 부호 및 방향 정보 모두와 관련된 태스크를 다룹니다. 제안된 스펙트럼 GNN이 부호 및 방향 정보를 모두 반영하는 데 효과적이며 광범위한 데이터셋에서 최고 수준의 성능을 달성함을 입증합니다. 추가로, 부호 방향 확률적 블록 모델(Signed Directed Stochastic Block Model)이라 부르는 새로운 합성 네트워크 모델(model)과 금융 시계열의 리드-래그(lead-lag) 관계에 기반한 다수의 새로운 실제 데이터셋을 제공합니다.

arXiv:2507.03209v2q-bio.QM공식 코드 · Papers with Code

리피드 나노입자(LNP) 전입 효율 예측을 위한 머신러닝 벤치마킹 프레임워크

A Machine Learning Benchmarking Framework for Lipid Nanoparticle Transfection Efficiency Prediction

효율적인 리피드 나노입자(LNP) 매개 RNA 전달을 위한 새로운 이온화성 리피드(ionizable lipid)의 발견은 RNA 치료제 개발에서 여전히 주요 병목으로 남아 있다. 최근 발전은 리피드 구조로부터 직접 전입(transfection) 효율을 예측할 수 있는 머신러닝(ML) 모델의 잠재력을 보여주며, 이는 고처리량 가상 스크리닝을 가능하게 하고 후보 물질(lead) 선별을 가속한다. 그러나 LNP 전입 예측을 위한 새로운 모델이 계속 등장함에 따라, 엄격하고 표준화된 벤치마킹의 부재는 상당한 위험을 초래하며, 발견(discovery) 목적에서의 신뢰성에 대한 확신을 약화시킬 수 있다. 여기서 우리는 이온화성 리피드 구조에 기반한 전입 예측 모델을 평가하기 위한 견고한 ML 벤치마킹 프레임워크를 제시한다. 이 프레임워크는 전통적인 모델, 순전파 신경망(feedforward neural network), 그리고 최신(state-of-the-art) 그래프 기반 방법에 이르는 광범위한 ML 아키텍처와 결합된 다양한 분자 표현(molecular representation)을 체계적으로 벤치마킹한다. 또한, 제시된 프레임워크는 모델 일반화(generalization) 평가를 지원하며, 표준 회귀(regression) 지표를 넘어 예측 신뢰성을 평가한다. Xu et al.이 최초로 보고한 HeLa 전입 데이터셋에서 파생된 1,100개의 고유 이온화성 리피드 구조로 구성된 큐레이션 데이터셋을 사용하여, 이 프레임워크 내에서 명시적 분자 부분구조(substructure) 인코딩을 활용하는 모델들이 지속적으로 최고의 예측 정확도를 달성하며, 보다 정교한 새로운 모델 개발을 위한 필수 베이스라인으로 기능해야 함을 보여준다. 반면, AGILE, Chemprop, KPGT를 포함한 일부 현재의 그래프 기반 모델들은 비교적으로 낮은 정확도를 보이는 경향이 있다. 제시된 프레임워크는 표준화되고 투명하며 포괄적인 벤치마킹 자원을 제공하여, 새롭게 등장하는 아키텍처의 의미 있는 비교를 가능하게 하고 리피드 기반 RNA 전달에서의 예측 모델 향후 개발을 위한 강력한 베이스라인을 확립한다.

arXiv:2405.19062v2cs.LG공식 코드 · Papers with Code

Invariant Graph Representations for Continuous-Time Dynamic Graphs Under Distribution Shifts

Continuous-Time Dynamic Graphs (CTDGs) enable fine-grained modeling of evolving relational systems. However, most existing CTDG representation learning methods are tailored to in-distribution settings and exhibit limited robustness under out-of-distribution (OOD) shifts. Although recent causal approaches learn invariant representations via interventions, they are primarily designed for static or discrete-time graphs and become computationally prohibitive for CTDGs due to the combinatorial explosion of structural and temporal variations. To address these challenges, we propose CIR, a framework grounded in a novel structural causal model termed the ICCM. To avoid exhaustive interventions, we leverage the Normalized Weighted Geometric Mean (NWGM) to efficiently approximate interventional predictions. We further instantiate ICCM within a practical deep learning architecture that jointly captures invariant structural and temporal patterns through dedicated subgraph extractors, and maintains an environment memory bank to model distributional shifts across evolving contexts. Extensive experiments demonstrate that CIR consistently outperforms existing methods under diverse OOD scenarios.

arXiv:2306.05239v3cs.CV공식 코드 · Papers with Code

Point-Voxel Absorbing Graph Representation Learning for Event Stream based Recognition

Sampled point and voxel methods are usually employed to downsample the dense events into sparse ones. After that, one popular way is to leverage a graph model which treats the sparse points/voxels as nodes and adopts graph neural networks (GNNs) to learn the representation of event data. Although good performance can be obtained, however, their results are still limited mainly due to two issues. (1) Existing event GNNs generally adopt the additional max (or mean) pooling layer to summarize all node embeddings into a single graph-level representation for the whole event data representation. However, this approach fails to capture the importance of graph nodes and also fails to be fully aware of the node representations. (2) Existing methods generally employ either a sparse point or voxel graph representation model which thus lacks consideration of the complementary between these two types of representation models. To address these issues, we propose a novel dual point-voxel absorbing graph representation learning for event stream data representation. To be specific, given the input event stream, we first transform it into the sparse event cloud and voxel grids and build dual absorbing graph models for them respectively. Then, we design a novel absorbing graph convolutional network (AGCN) for our dual absorbing graph representation and learning. The key aspect of the proposed AGCN is its ability to effectively capture the importance of nodes and thus be fully aware of node representations in summarizing all node representations through the introduced absorbing nodes. Extensive experiments on multiple event-based classification benchmark datasets fully validated the effectiveness of our framework.