Dataset for our paper Disambiguation-Centric Finetuning Makes Enterprise Tool-Calling LLMs More Realistic and Less Risky which includes 5000 enterprise tools and the corresponding dialogues generated using DiaFORGE UTC data engine. The dataset is generated with the data generation engine. The engine simulates a user agent and an assistant agent in a dialogue, where the user agent has a persona and the assistant agent has access to a set of tools. Detailed information about the data generation process can be found in the paper. Each entry in the dataset contains: - seed: Gold tool for the generated dialogue. - user_persona: Persona of the simulated user agent. - messages: List of messages in the dialogue, where each message is a dictionary containing: - distractor_tools: List of distractor tools that are not the gold tool but are relevant to the dialogue. These tools are used by user agent to generate utterances that are hard to disambiguate from the gold tool. - retrieved_tools: List of tools retrieved by the assistant agent and used by the assistant agent in order to ask clarifying questions to the user agent.
Finnish chat conversation corpus and includes unscripted conversations on seven topics from people of different ages. Source: FinChat: Corpus and evaluation setup for Finnish chat conversations on everyday topics
Harry Potter Dialogue is the first dialogue dataset that integrates with scene, attributes and relations which are dynamically changed as the storyline goes on. Our work can facilitate research to construct more human-like conversational systems in practice. For example, virtual assistant, NPC in games, etc. Moreover, HPD can both support dialogue generation and retrieval tasks.
JDDC 2.0 is a large-scale multimodal multi-turn dialogue dataset collected from a mainstream Chinese E-commerce platform JD.com, containing about 246 thousand dialogue sessions, 3 million utterances, and 507 thousand images, along with product knowledge bases and image category annotations. The dataset is divided into the training set, the validation set, and the test set according to the ratio of 80%, 10%, and 10%.
To evaluate our proposed strategy of asynchronous communication for LLMs, we run games of Mafia with human players, incorporating an LLM-based agent as an additional player, within an asynchronous chat environment. The data includes all players messages and votes including timestamps, game management messages (e.g., announcements of the beginning and end of phases), in addition to records related to the agent, such as the prompts that were provided at each timestamp. While prior datasets for Mafia exist, ours is the first to integrate an LLM agent, allowing analysis of human-LLM interactions. Detailed analysis of the data is available in our paper: Time to Talk: LLM Agents for Asynchronous Group Communication in Mafia Games
Dataset Description Our dataset contains questions from a well-known software testing book Introduction to Software Testing 2nd Edition by Ammann and Offutt. We use all the text-book questions in Chapters 1 to 5 that have solutions available on the book’s official website. Our dataset contains 40 such questions from these five chapters. 31 questions out of the 40 are multipart questions and the rest 9 are independent. This tool generates responses from the ChatGPT automatically for these questions. All of these questions are run in both shared and separate context. More information about the contexts can be found below. ### Combined.xlsx Contains all the questions & answers for 3 iterations of both shared and separate contexts. Contains labels for answers and explanations given by ChatGPT. ### Combined_pair.xlsx Contains the same data as combined.xlsx except for the questions that are independent i.e., not part of a multipart question. ### Combined_analysis.xlsx Contains the result and analysis of the four research questions. Besides, it contains various illustrations for the results. ### Combined-temp.xlsx Questions with missing shared contexts are replaced with the answers for the separate context to easily fetch the data for RQ2 & RQ3 from a single column. ### examples folders Contains examples of some interesting response categories. ### Case Study.pdf Contains the following analysis: - When responses are likely to be incorrect? - What are the reasons for being incorrect? - Can we fix it with prompt engineering? - Case studies with actual examples. ## Separate Context Query In separate context queries, we treat each of the 31 multipart questions as an independent question. Each sub-question is asked in a separate chat thread. Combining with the nine independent questions, a total of 40 questions are asked for each run. To evaluate the consistency in ChatGPT’s responses, we collect a total of three runs for each question, which results in a total of 120 responses from ChatGPT. ## Shared Context Query Our dataset contains six questions that contain total 31 multipart questions or sub-questions and nine questions that do not. These six sub-questions are asked in a chat thread that are shared with other sub-questions as long as the sub-questions refer to the same code or scenario.
The Customer Support on Twitter dataset is a large, modern corpus of tweets and replies to aid innovation in natural language understanding and conversational models, and for study of modern customer support practices and impact.