PyTorchKR Papers

최신 논문

총 27편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:2005.11014공식 코드 · Papers with Code

Intent Mining from past conversations for conversational agent

Conversational systems are of primary interest in the AI community. Chatbots are increasingly being deployed to provide round-the-clock support and to increase customer engagement. Many of the commercial bot building frameworks follow a standard approach that requires one to build and train an intent model to recognize a user input. Intent models are trained in a supervised setting with a collection of textual utterance and intent label pairs. Gathering a substantial and wide coverage of training data for different intent is a bottleneck in the bot building process. Moreover, the cost of labeling a hundred to thousands of conversations with intent is a time consuming and laborious job. In this paper, we present an intent discovery framework that involves 4 primary steps: Extraction of textual utterances from a conversation using a pre-trained domain agnostic Dialog Act Classifier (Data Extraction), automatic clustering of similar user utterances (Clustering), manual annotation of clusters with an intent label (Labeling) and propagation of intent labels to the utterances from the previous step, which are not mapped to any cluster (Label Propagation); to generate intent training data from raw conversations. We have introduced a novel density-based clustering algorithm ITER-DBSCAN for unbalanced data clustering. Subject Matter Expert (Annotators with domain expertise) manually looks into the clustered user utterances and provides an intent label for discovery. We conducted user studies to validate the effectiveness of the trained intent model generated in terms of coverage of intents, accuracy and time saving concerning manual annotation. Although the system is developed for building an intent model for the conversational system, this framework can also be used for a short text clustering or as a labeling framework.

arXiv:2001.11631공식 코드 · Papers with Code

Enhancement of Short Text Clustering by Iterative Classification

Short text clustering is a challenging task due to the lack of signal contained in such short texts. In this work, we propose iterative classification as a method to b o ost the clustering quality (e.g., accuracy) of short texts. Given a clustering of short texts obtained using an arbitrary clustering algorithm, iterative classification applies outlier removal to obtain outlier-free clusters. Then it trains a classification algorithm using the non-outliers based on their cluster distributions. Using the trained classification model, iterative classification reclassifies the outliers to obtain a new set of clusters. By repeating this several times, we obtain a much improved clustering of texts. Our experimental results show that the proposed clustering enhancement method not only improves the clustering quality of different clustering methods (e.g., k-means, k-means--, and hierarchical clustering) but also outperforms the state-of-the-art short text clustering methods on several short text datasets by a statistically significant margin.

arXiv:1912.03720

Attentive Representation Learning with Adversarial Training for Short Text Clustering

Short text clustering has far-reaching effects on semantic analysis, showing its importance for multiple applications such as corpus summarization and information retrieval. However, it inevitably encounters the severe sparsity of short text representations, making the previous clustering approaches still far from satisfactory. In this paper, we present a novel attentive representation learning model for shot text clustering, wherein cluster-level attention is proposed to capture the correlations between text representations and cluster representations. Relying on this, the representation learning and clustering for short texts are seamlessly integrated into a unified model. To further ensure robust model training for short texts, we apply adversarial training to the unsupervised clustering setting, by injecting perturbations into the cluster representations. The model parameters and perturbations are optimized alternately through a minimax game. Extensive experiments on four real-world short text datasets demonstrate the superiority of the proposed model over several strong competitors, verifying that robust adversarial training yields substantial performance gains.

arXiv:1911.08891공식 코드 · Papers with Code

Discovering New Intents via Constrained Deep Adaptive Clustering with Cluster Refinement

Identifying new user intents is an essential task in the dialogue system. However, it is hard to get satisfying clustering results since the definition of intents is strongly guided by prior knowledge. Existing methods incorporate prior knowledge by intensive feature engineering, which not only leads to overfitting but also makes it sensitive to the number of clusters. In this paper, we propose constrained deep adaptive clustering with cluster refinement (CDAC+), an end-to-end clustering method that can naturally incorporate pairwise constraints as prior knowledge to guide the clustering process. Moreover, we refine the clusters by forcing the model to learn from the high confidence assignments. After eliminating low confidence assignments, our approach is surprisingly insensitive to the number of clusters. Experimental results on the three benchmark datasets show that our method can yield significant improvements over strong baselines.

arXiv:1701.00185공식 코드 · Papers with Code

Self-Taught Convolutional Neural Networks for Short Text Clustering

Short text clustering is a challenging problem due to its sparseness of text representation. Here we propose a flexible Self-Taught Convolutional neural network framework for Short Text Clustering (dubbed STC^2), which can flexibly and successfully incorporate more useful semantic features and learn non-biased deep text representation in an unsupervised manner. In our framework, the original raw text features are firstly embedded into compact binary codes by using one existing unsupervised dimensionality reduction methods. Then, word embeddings are explored and fed into convolutional neural networks to learn deep feature representations, meanwhile the output units are used to fit the pre-trained binary codes in the training process. Finally, we get the optimal clusters by employing K-means to cluster the learned representations. Extensive experimental results demonstrate that the proposed framework is effective, flexible and outperform several popular clustering methods when tested on three public short text datasets.

arXiv:1602.06797

Semi-supervised Clustering for Short Text via Deep Representation Learning

In this work, we propose a semi-supervised method for short text clustering, where we represent texts as distributed vectors with neural networks, and use a small amount of labeled data to specify our intention for clustering. We design a novel objective to combine the representation learning process and the k-means clustering process together, and optimize the objective with both labeled data and unlabeled data iteratively until convergence through three steps: (1) assign each short text to its nearest centroid based on its representation from the current neural networks; (2) re-estimate the cluster centroids based on cluster assignments from step (1); (3) update neural networks according to the objective by keeping centroids and cluster assignments fixed. Experimental results on four datasets show that our method works significantly better than several other text clustering methods.

arXiv:2607.10548v1stat.ML

위만 보지 말고, 주변을 살펴보라: 단문 텍스트 클러스터링을 위한 최적 수송에서 전역 구조와 지역 일관성의 조화

Beyond Looking Up, Try Looking Around: Harmonizing Global Structure and Local Consistency in Optimal Transport for Short Text Clustering

최적 수송(Optimal Transport, OT) 기반 의사 레이블링(pseudo-labeling)은 단문 텍스트 클러스터링 향상을 위한 효과적인 메커니즘으로 자리잡았다. 기존 OT 방법들은 샘플 간 의미적 일관성 모델링이 부족하여 의미적으로 유사한 샘플들에 서로 다른 의사 레이블을 할당할 수 있다. 이러한 잘못된 의사 레이블은 모델이 열등한 클러스터를 생성하게 만들 수 있다. 본 논문은 기존 OT 방법에서 간과된 의미적 일관성 문제를 해결하고 신뢰할 수 있는 의사 레이블을 생성하여 클러스터링을 촉진하는 새로운 단문 텍스트 클러스터링 프레임워크를 제안한다. 구체적으로, 제안된 접근법은 샘플 간 의미적 관계를 포착하기 위한 인스턴스 수준 어텐션(attention) 메커니즘을 설계하고, 이를 OT 공식화에 통합하여 수송 과정에 이웃 의미 인식(neighborhood semantic awareness) 능력을 부여한다. 제안된 OT 공식화를 풀어냄으로써, 샘플 간 의미적 일관성과 샘플-클러스터 전역 구조 정보를 동시에 고려하는 신뢰할 수 있는 의사 레이블이 획득된다. 이 의사 레이블들은 모델(model)이 정확한 클러스터링을 달성하도록 안내하는 감독 신호로 활용된다. 광범위한 실험을 통해 제안된 접근법이 최신 방법들을 능가함을 입증한다. 코드는 https://github.com/YZH0905/CAOT-STC 에서 확인할 수 있다.