Conversational Response Selection

The Reddit dataset is a graph dataset from Reddit posts made in the month of September, 2014. The node label in this case is the community, or “subreddit”, that a post belongs to. 50 large communities have been sampled to build a post-to-post graph, connecting posts if the same user comments on both. In total this dataset contains 232,965 posts with an average degree of 492. The first 20 days are used for training and the remaining days for testing (with 30% used for validation). For features, off-the-shelf 300-dimensional GloVe CommonCrawl word vectors are used. Source: https://arxiv.org/pdf/1706.02216.pdf Image Source: https://minimaxir.com/2016/05/reddit-graph/

We release Douban Conversation Corpus, comprising a training data set, a development set and a test set for retrieval based chatbot. The statistics of Douban Conversation Corpus are shown in the following table. | |Train|Val| Test | | ------------- |:-------------:|:-------------:|:-------------:| | session-response pairs | 1m|50k| 10k | | Avg. positive response per session | 1|1| 1.18 | | Fless Kappa | N\A|N\A|0.41 | | Min turn per session | 3|3| 3 | | Max ture per session | 98|91|45 | | Average turn per session | 6.69|6.75|5.95 | | Average Word per utterance | 18.56|18.50|20.74 | The test data contains 1000 dialogue context, and for each context we create 10 responses as candidates. We recruited three labelers to judge if a candidate is a proper response to the session. A proper response means the response can naturally reply to the message given the context. Each pair received three labels and the majority of the labels was taken as the final decision. <br> As far as we known, this is the first human-labeled test set for retrieval-based chatbots. The entire corpus link https://www.dropbox.com/s/90t0qtji9ow20ca/DoubanConversaionCorpus.zip?dl=0

Ubuntu Dialogue Corpus (UDC) is a dataset containing almost 1 million multi-turn dialogues, with a total of over 7 million utterances and 100 million words. This provides a unique resource for research into building dialogue managers based on neural language models that can make use of large amounts of unlabeled data. The dataset has both the multi-turn property of conversations in the Dialog State Tracking Challenge datasets, and the unstructured nature of interactions from microblog services such as Twitter. Source: The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems

We release E-commerce Dialogue Corpus, comprising a training data set, a development set and a test set for retrieval based chatbot. The statistics of E-commerical Conversation Corpus are shown in the following table. | |Train|Val| Test | | ------------- |:-------------:|:-------------:|:-------------:| | Session-response pairs | 1m|10k| 10k | | Avg. positive response per session|1|1|1| | Min turn per session|3|3|3| | Max ture per session|10|10|10| | Average turn per session|5.51|5.48|5.64 | Average Word per utterance|7.02|6.99|7.11 The full corpus can be downloaded from https://drive.google.com/file/d/154J-neBo20ABtSmJDvm7DK0eTuieAuvw/view?usp=sharing.

The Ubuntu IRC dataset is a valuable resource for research in natural language understanding and dialogue systems. Let me provide you with some details: 1. Ubuntu Dialogue Corpus: - This dataset contains almost 1 million multi-turn dialogues, comprising over 7 million utterances and 100 million words. - It serves as a unique resource for building dialogue managers based on neural language models that can leverage large amounts of unlabeled data³. - The dialogues are sourced from IRC (Internet Relay Chat) conversations related to Ubuntu, a popular open-source operating system. - Researchers can use this corpus to explore various aspects of dialogue understanding and generation. 2. Specifics of the Ubuntu IRC Dataset: - The dataset includes 77,563 annotated messages from IRC. - Most of these messages originate from the Ubuntu IRC Logs for the #ubuntu channel. - Additionally, a smaller subset is a re-annotation of data from the #linux channel, which was originally collected by Elsner and Charniak in 2008². - You can find this dataset in the kummerfeld/data folder of the repository². In summary, the Ubuntu IRC dataset provides a rich collection of dialogues that researchers can use to advance the field of natural language processing and dialogue modeling. 🌐🗣️ (1) The Ubuntu Dialogue Corpus: A Large Dataset for Research in .... https://arxiv.org/abs/1506.08909. (2) GitHub - amarazad/DSRNet: Meta-Context Transformers for Domain-Specific .... https://github.com/amarazad/DSRNet. (3) Ubuntu Dialogue Corpus | Kaggle. https://www.kaggle.com/datasets/rtatman/ubuntu-dialogue-corpus. (4) GitHub - jkkummerfeld/irc-disentanglement: Dataset and model for .... https://github.com/jkkummerfeld/irc-disentanglement.

We release Douban Conversation Corpus, comprising a training data set, a development set and a test set for retrieval based chatbot. The statistics of Douban Conversation Corpus are shown in the following table. | |Train|Val| Test | | ------------- |:-------------:|:-------------:|:-------------:| | session-response pairs | 1m|50k| 10k | | Avg. positive response per session | 1|1| 1.18 | | Fless Kappa | N\A|N\A|0.41 | | Min turn per session | 3|3| 3 | | Max ture per session | 98|91|45 | | Average turn per session | 6.69|6.75|5.95 | | Average Word per utterance | 18.56|18.50|20.74 | The test data contains 1000 dialogue context, and for each context we create 10 responses as candidates. We recruited three labelers to judge if a candidate is a proper response to the session. A proper response means the response can naturally reply to the message given the context. Each pair received three labels and the majority of the labels was taken as the final decision. <br> As far as we known, this is the first human-labeled test set for retrieval-based chatbots. The entire corpus link https://www.dropbox.com/s/90t0qtji9ow20ca/DoubanConversaionCorpus.zip?dl=0 ## Data template label \t conversation utterances (splited by \t) \t response

The DSTC7 Task 1 dataset is a dataset and task for goal-oriented dialogue. The data originates from human-human conversations, which is built from online resources, specifically the Ubuntu Internet Relay Chat (IRC) channel and an Advising dataset from the University of Michigan. Source: Multimodal Transformer Networks for End-to-End Video-Grounded Dialogue Systems Image Source: https://www.aclweb.org/anthology/W19-4107.pdf

| | Train | Validation | Test | Ranking Test | | --------- | ----- | ---------- | ------- | ------------ | | size | 0.4M | 50K | 5K | 800 | | pos:neg | 1:1 | 1:9 | 1.2:8.8 | - | | avg turns | 5.0 | 5.0 | 5.0 | 5.0 | Ranking test set contains the high-quality responses that selected by some baselines, and their correlation with the conversation context are carefully annotated by 8 professional annotators (the average annotation scores are saved for ranking). For ranking test set, the metrics should be NDCG@3 and NDCG@5, since the correlation scores are provided. More details are available in the Appendix of the paper.

Reddit Corpus is part of a repository of conversational datasets consisting of hundreds of millions of examples, and a standardised evaluation procedure for conversational response selection models using '1-of-100 accuracy'. The Reddit Corpus contains 726 million multi-turn dialogues from the Reddit board. Source: conversational-datasets Image source: Henderson et al.

| | Train | Validation | Test | Ranking Test | | --------- | ----- | ---------- | ------- | ------------ | | size | 0.4M | 50K | 5K | 800 | | pos:neg | 1:1 | 1:9 | 1.2:8.8 | - | | avg turns | 5.0 | 5.0 | 5.0 | 5.0 | Ranking test set contains the high-quality responses that selected by some baselines, and their correlation with the conversation context are carefully annotated by 8 professional annotators (the average annotation scores are saved for ranking). For ranking test set, the metrics should be NDCG@3 and NDCG@5, since the correlation scores are provided. More details are available in the Appendix of the paper.

Advising Corpus is a dataset based on an entirely new collection of dialogues in which university students are being advised which classes to take. These were collected at the University of Michigan with IRB approval. They were released as part of DSTC 7 track 1 and used again in DSTC 8 track 2. Source: Gunasekara et al. Image source: Gunasekara et al.

This dataset is for evaluating the task of Black-box Multi-agent Integration which focuses on combining the capabilities of multiple black-box conversational agents at scale. It provides data to explore two main frameworks of exploration: question agent pairing and question response pairing. Overall this dataset contains 5550 utterances with 19 question-response pairs per question (one from each of the 19 agents), 105,450 in total across 37 domains. The utterances are split into 3700 utterances (100 examples per domain) for the training set and 1850 (50 per domain) for the test set. The train and test sets respectively contain 2399 and 1186 utterances with at least one positive question-response pair. In the remaining examples, none of the agents were able to achieve annotator agreement (>= 3).