In this document I present an approach to answer validation and reranking for question answering (QA) systems. A cased-based reasoning (CBR) system judges answer candidates for questions from annotated answer candidates for earlier questions. The promise of this approach is that user feedback will result in improved answers of the QA system, due to the growing case base. In the paper, I present the adequate structuring of the case base and the appropriate selection of relevant similarity measures, in order to solve the answer validation problem. The structural case base is built from annotated MultiNet graphs, which provide representations for natural language expressions, and corresponding graph similarity measures. I cover a priori relations to experienced answer candidates for former questions. I compare the CBR System results to current approaches in an experiment integrating CBR into an existing framework for answer validation and reranking. This integration is achieved by adding CBR-related features to the input of a learned ranking model that determines the final answer ranking. In the experiments based on QA@CLEF questions, the best learned models make heavy use of CBR features. Observing the results with a continually growing case base, I present a positive effect of the size of the case base on the accuracy of the CBR subsystem.
Graph theory has drawn a lot of attention in the field of Neuroscience during the last decade, mainly due to the abundance of tools that it provides to explore the interactions of elements in a complex network like the brain. The local and global organization of a brain network can shed light on mechanisms of complex cognitive functions, while disruptions within the network can be linked to neurodevelopmental disorders. In this effort, the construction of a representative brain network for each individual is critical for further analysis. Additionally, graph comparison is an essential step for inference and classification analyses on brain graphs. In this work we explore a method based on graph edit distance for evaluating graph similarity, when correspondences between network elements are unknown due to different underlying subdivisions of the brain. We test this method on 30 unrelated subjects as well as 40 twin pairs and show that this method can accurately reflect the higher similarity between two related networks compared to unrelated ones, while identifying node correspondences.
Evaluating similarity between graphs is of major importance in several computer vision and pattern recognition problems, where graph representations are often used to model objects or interactions between elements. The choice of a distance or similarity metric is, however, not trivial and can be highly dependent on the application at hand. In this work, we propose a novel metric learning method to evaluate distance between graphs that leverages the power of convolutional neural networks, while exploiting concepts from spectral graph theory to allow these operations on irregular graphs. We demonstrate the potential of our method in the field of connectomics, where neuronal pathways or functional connections between brain regions are commonly modelled as graphs. In this problem, the definition of an appropriate graph similarity function is critical to unveil patterns of disruptions associated with certain brain disorders. Experimental results on the ABIDE dataset show that our method can learn a graph similarity metric tailored for a clinical application, improving the performance of a simple k-nn classifier by 11.9% compared to a traditional distance metric.
In this paper, we propose a new type of graph, denoted as "embedded-graph", and its theory, which employs a distributed representation to describe the relations on the graph edges. Embedded-graphs can express linguistic and complicated relations, which cannot be expressed by the existing edge-graphs or weighted-graphs. We introduce the mathematical definition of embedded-graph, translation, edge distance, and graph similarity. We can transform an embedded-graph into a weighted-graph and a weighted-graph into an edge-graph by the translation method and by threshold calculation, respectively. The edge distance of an embedded-graph is a distance based on the components of a target vector, and it is calculated through cosine similarity with the target vector. The graph similarity is obtained considering the relations with linguistic complexity. In addition, we provide some examples and data structures for embedded-graphs in this paper.
Convolutional neural networks (CNNs) can be applied to graph similarity matching, in which case they are called graph CNNs. Graph CNNs are attracting increasing attention due to their effectiveness and efficiency. However, the existing convolution approaches focus only on regular data forms and require the transfer of the graph or key node neighborhoods of the graph into the same fixed form. During this transfer process, structural information of the graph can be lost, and some redundant information can be incorporated. To overcome this problem, we propose the disordered graph convolutional neural network (DGCNN) based on the mixed Gaussian model, which extends the CNN by adding a preprocessing layer called the disordered graph convolutional layer (DGCL). The DGCL uses a mixed Gaussian function to realize the mapping between the convolution kernel and the nodes in the neighborhood of the graph. The output of the DGCL is the input of the CNN. We further implement a backward-propagation optimization process of the convolutional layer by which we incorporate the feature-learning model of the irregular node neighborhood structure into the network. Thereafter, the optimization of the convolution kernel becomes part of the neural network learning process. The DGCNN can accept arbitrary scaled and disordered neighborhood graph structures as the receptive fields of CNNs, which reduces information loss during graph transformation. Finally, we perform experiments on multiple standard graph datasets. The results show that the proposed method outperforms the state-of-the-art methods in graph classification and retrieval.
Digital presence in the world of online social media entails significant privacy risks. In this work we consider a privacy threat to a social network in which an attacker has access to a subset of random walk-based node similarities, such as effective resistances (i.e., commute times) or personalized PageRank scores. Using these similarities, the attacker's goal is to infer as much information as possible about the underlying network, including any remaining unknown pairwise node similarities and edges. For the effective resistance metric, we show that with just a small subset of measurements, the attacker can learn a large fraction of edges in a social network, even when the measurements are noisy. We also show that it is possible to learn a graph which accurately matches the underlying network on all other effective resistances. This second observation is interesting from a data mining perspective, since it can be expensive to accurately compute all effective resistances. As an alternative, our graphs learned from just a subset of approximate effective resistances can be used as surrogates in a wide range of applications that use effective resistances to probe graph structure, including for graph clustering, node centrality evaluation, and anomaly detection. We obtain our results by formalizing the graph learning objective mathematically, using two optimization problems. One formulation is convex and can be solved provably in polynomial time. The other is not, but we solve it efficiently with projected gradient and coordinate descent. We demonstrate the effectiveness of these methods on a number of social networks obtained from Facebook. We also discuss how our methods can be generalized to other random walk-based similarities, such as personalized PageRank. Our code is available at https://github.com/cnmusco/graph-similarity-learning.
Unsupervised machine translation---i.e., not assuming any cross-lingual supervision signal, whether a dictionary, translations, or comparable corpora---seems impossible, but nevertheless, Lample et al. (2018) recently proposed a fully unsupervised machine translation (MT) model. The model relies heavily on an adversarial, unsupervised alignment of word embedding spaces for bilingual dictionary induction (Conneau et al., 2018), which we examine here. Our results identify the limitations of current unsupervised MT: unsupervised bilingual dictionary induction performs much worse on morphologically rich languages that are not dependent marking, when monolingual corpora from different domains or different embedding algorithms are used. We show that a simple trick, exploiting a weak supervision signal from identical words, enables more robust induction, and establish a near-perfect correlation between unsupervised bilingual dictionary induction performance and a previously unexplored graph similarity metric.
Spectral clustering is one of the most effective clustering approaches that capture hidden cluster structures in the data. However, it does not scale well to large-scale problems due to its quadratic complexity in constructing similarity graphs and computing subsequent eigendecomposition. Although a number of methods have been proposed to accelerate spectral clustering, most of them compromise considerable information loss in the original data for reducing computational bottlenecks. In this paper, we present a novel scalable spectral clustering method using Random Binning features (RB) to simultaneously accelerate both similarity graph construction and the eigendecomposition. Specifically, we implicitly approximate the graph similarity (kernel) matrix by the inner product of a large sparse feature matrix generated by RB. Then we introduce a state-of-the-art SVD solver to effectively compute eigenvectors of this large matrix for spectral clustering. Using these two building blocks, we reduce the computational cost from quadratic to linear in the number of data points while achieving similar accuracy. Our theoretical analysis shows that spectral clustering via RB converges faster to the exact spectral clustering than the standard Random Feature approximation. Extensive experiments on 8 benchmarks show that the proposed method either outperforms or matches the state-of-the-art methods in both accuracy and runtime. Moreover, our method exhibits linear scalability in both the number of data samples and the number of RB features.
The von Neumann graph entropy (VNGE) facilitates measurement of information divergence and distance between graphs in a graph sequence. It has been successfully applied to various learning tasks driven by network-based data. While effective, VNGE is computationally demanding as it requires the full eigenspectrum of the graph Laplacian matrix. In this paper, we propose a new computational framework, Fast Incremental von Neumann Graph EntRopy (FINGER), which approaches VNGE with a performance guarantee. FINGER reduces the cubic complexity of VNGE to linear complexity in the number of nodes and edges, and thus enables online computation based on incremental graph changes. We also show asymptotic equivalence of FINGER to the exact VNGE, and derive its approximation error bounds. Based on FINGER, we propose efficient algorithms for computing Jensen-Shannon distance between graphs. Our experimental results on different random graph models demonstrate the computational efficiency and the asymptotic equivalence of FINGER. In addition, we apply FINGER to two real-world applications and one synthesized anomaly detection dataset, and corroborate its superior performance over seven baseline graph similarity methods.
현대의 대규모 언어 모델(LLM)은 복잡한 임상 사례 벤치마크에서 60~70%의 진단 정확도에 도달하지만, 정확도만으로는 안정적인 임상 기반 추론과 패턴 매칭을 구별할 수 없다. 우리는 5가지 노드 유형과 7가지 엣지 유형을 갖는 도메인 기반 온톨로지를 사용하여 자유 형식의 LLM 진단 추적에서 추출한 구조적 그래프 표현인 임상 추론 그래프를 소개한다. 이 파이프라인을 50개의 New England Journal of Medicine 임상병리학 컨퍼런스 사례와 세 가지 프롬프트 조건에 걸쳐 다섯 개 모델에서 생성된 750개의 추적에 적용하고, 임상적으로 유사한 사례들에서 진단 추적이 안정적인 구조적 추론 패턴, 즉 진단 스키마를 보이는지 검증하였다. 이를 임상적으로 유사한 사례 쌍 간의 그래프 유사도가 임상적으로 유사하지 않은 사례 쌍보다 높은지로 조작적 정의하였다. 15개의 모델-조건 비교 전반에 걸쳐 클러스터 내 및 클러스터 간 복합 유사도는 거의 동등하였으며, 다중 검정 보정을 통과한 비교는 없었다. 구성 요소 수준 분석에서도 잔여 콘텐츠 신호는 스키마 수준에 크게 미치지 못하였다. 그래프 유사도는 두 모델이 모두 정답인 쌍(0.488)과 모두 오답인 쌍(0.484)에서도 거의 동일하여, 그래프 구조가 진단 정확도에 반영되지 않는 차원을 포착함을 시사한다. 구조적 반성 프롬프트는 추적 내 명시적 판별 특징 분석을 증가시켰으나(+33%), 사례 간 일관성은 높이지 못하였다. 이러한 결과는 스키마 수준의 추론 일관성 없이도 진단 역량이 존재함을 보여주며, 최종 답변 정확도가 프로세스 수준 평가로 보완되어야 함을 나타낸다. 우리는 LLM 임상 추론의 구조적 평가를 위한 자원으로서 온톨로지, 추출 파이프라인, 검증 프로토콜, 그리고 추출된 추론 그래프와 유사도 결과물을 공개한다.