Data Integration

DBP15k contains four language-specific KGs that are respectively extracted from English (En), Chinese (Zh), French (Fr) and Japanese (Ja) DBpedia, each of which contains around 65k-106k entities. Three sets of 15k alignment labels are constructed to align entities between each of the other three languages and En. Source: Cross-lingual Entity Alignment for Knowledge Graphs with Incidental Supervision from Free Text

MMKG is a collection of three knowledge graphs for link prediction and entity matching research. Contrary to other knowledge graph datasets, these knowledge graphs contain both numerical features and images for all entities as well as entity alignments between pairs of KGs. While MMKG is intended to perform relational reasoning across different entities and images, previous resources are intended to perform visual reasoning within the same image. The three knowledge graphs augmented with numerical features and images are called FB15k, YAGO15k, and DBPEDIA15k.

The Amazon-Google dataset for entity resolution derives from the online retailers Amazon.com and the product search service of Google accessible through the Google Base Data API. The dataset contains 1363 entities from amazon.com and 3226 google products as well as a gold standard (perfect mapping) with 1300 matching record pairs between the two data sources. The common attributes between the two data sources are: product name, product description, manufacturer and price. The dataset was initially published in the repository of the Database Group of the University of Leipzig: https://dbs.uni-leipzig.de/research/projects/object_matching/benchmark_datasets_for_entity_resolution To enable the reproducibility of the results and the comparability of the performance of different matchers on the Amazon-Google matching task, the dataset was split into fixed train, validation and test sets. The fixed splits are provided in the CompERBench repository: http://data.dws.informatik.uni-mannheim.de/benchmarkmatchingtasks/index.html

The Abt-Buy dataset for entity resolution derives from the online retailers Abt.com and Buy.com. The dataset contains 1081 entities from abt.com and 1092 entities from buy.com as well as a gold standard (perfect mapping) with 1097 matching record pairs between the two data sources. The common attributes between the two data sources are: product name, product description and product price. The dataset was initially published in the repository of the Database Group of the University of Leipzig: https://dbs.uni-leipzig.de/research/projects/object_matching/benchmark_datasets_for_entity_resolution To enable the reproducibility of the results and the comparability of the performance of different matchers on the Abt-Buy matching task, the dataset was split into fixed train, validation and test sets. The fixed splits are provided in the CompERBench repository: http://data.dws.informatik.uni-mannheim.de/benchmarkmatchingtasks/index.html

GitTables is a corpus of currently 1M relational tables extracted from CSV files in GitHub covering 96 topics. Table columns in GitTables have been annotated with more than 2K different semantic types from Schema.org and DBpedia. The column annotations consist of semantic types, hierarchical relations, range types, table domain and descriptions. The tables were annotated using two methods: the semantic method and syntactic one. This leads to two kinds of annotations which in the metadata of the tables are referred to as syntactic and semantic annotations. The first method annotated 888,678 tables with Schema.org semantic types and 875,630 with DBpedia, while the second method annotated 1,161,117 tables with Schema.org and 1,156,601 with DBpedia semantic types. Some statistics about the tables are provided in the table below, "Columns" referring to the number of annotated columns and "Classes" to the number of unique DBpedia or Schema.org semantic types used for annotation. | | Columns | Classes | |----------------------|------------|---------| | Syntactic-DBpedia | 3,441,251 | 834 | | Syntactic-Schema.org | 2,671,588 | 677 | | Semantic-DBpedia | 10,757,184 | 2,380 | | Semantic-Schema.org | 10,475,155 | 2,407 |

The T2Dv2 dataset consists of 779 tables originating from the English-language subset of the WebTables corpus. 237 tables are annotated for the Table Type Detection task, 236 for the Columns Property Annotation (CPA) task and 235 for the Row Annotation task. The annotations that are used are DBpedia types, properties and entities. A subset of this dataset was annotated by Chen et al. for the Column Type Annotation (CTA) task where they annotate 236 tables with DBpedia types. In the papers it was used the subset was divided into training and testing splits and the evaluation was done on the testing split T2D-Te. This subset is available for download at their official Github repository. Some characteristics for the different tasks are provided in the table below, where "Annotations" refers to the number of cells/rows/columns/tables annotated and "Classes" to the number of unique classes used for annotation. | | Annotations | Classes | |----------------|---------|---------| | Column Property Annotation | 670 | 119 | | Row Annotation | 26,106 | 13,975 | | Table Type | 237 | 41 | | Column Type Annotation | 411 | 37 |

The ToughTables (2T) dataset was created for the SemTab challenge and includes 180 tables in total. The tables in this dataset can be categorized in two groups: the control (CTRL) group tables and tough (TOUGH) group tables. The CTRL group contains 60 tables generated by querying the DBpedia SPARQL endpoint and tables collected from Wikipedia and their characteristic is that they are easy to annotate. The TOUGH group contains 120 tables mainly scraped from the web, some containing misspelled words and nicknames/homonyms and their characteristic is that they are hard to annotate. In both groups some tables were generated by the authors where they added noise to the collected tables. The dataset was annotated for two tasks using DBpedia (DBP) types and entities and WikiData (WD): Column Type Annotation (CTA) and Cell Entity Annotation (CEA). In the table below the number of columns annotated for the CTA and number of cells annotated for the CEA task as well as the number of classes used are listed. | | Annotations| Classes | |-----|---------|---------| | DBP-Column Type Annotation | 540 | 39 | | DBP-Cell Entity Annotation | 663,656 | 16,023 | | WD-Column Type Annotation| 540 | 276 | | WD-Cell Entity Annotation | 667,244 | 24,653 |

The BioDiv dataset includes manually labeled tables for CTA and CEA from the biodiversity domain.

Many e-shops have started to mark-up product data within their HTML pages using the schema.org vocabulary. The Web Data Commons project regularly extracts such data from the Common Crawl, a large public web crawl. The Web Data Commons Training and Test Sets for Large-Scale Product Matching contain product offers from different e-shops in the form of binary product pairs (with corresponding label "match" or "no match") for four product categories, computers, cameras, watches and shoes. In order to support the evaluation of machine learning-based matching methods, the data is split into training, validation and test sets. For each product category, we provide training sets in four different sizes (2.000-70.000 pairs). Furthermore there are sets of ids for each training set for a possible validation split (stratified random draw) available. The test set for each product category consists of 1.100 product pairs. The labels of the test sets were manually checked while those of the training sets were derived using shared product identifiers from the Web via weak supervision. The data stems from the WDC Product Data Corpus for Large-Scale Product Matching - Version 2.0 which consists of 26 million product offers originating from 79 thousand websites.

DBLP Temporal is a dataset for temporal entity resolution, based on author profiles extracted from the Digital Bibliography and Library Project (DBLP).

SOTAB V2 features two annotation tasks: Column Type Annotation (CTA) and Columns Property Annotation (CPA). The goal of the Column Type Annotation (CTA) task is to annotate the columns of a table using 82 types from the Schema.org vocabulary, such as telephone, Duration, Mass, or Organization. The goal of the Columns Property Annotation (CPA) task is to annotate pairs of table columns with one out of 108 Schema.org properties, such as gtin, startDate, priceValidUntil, or recipeIngredient. The benchmark consists of 45,834 tables annotated for CTA and 30,220 tables annotated for CPA originating from 55,511 different websites. The tables are split into training-, validation- and test sets for both tasks. The tables cover 17 popular Schema.org types including Product, LocalBusiness, Event, and JobPosting. Some characteristics for the different tasks are provided in the table below, where "Columns" refers to the number of columns/column pairs labeled and "Classes" to the number of unique classes used for annotation. | | Train | | Validation | | Test | | Classes | |-----|--------|---------|------------|---------|--------|---------|---------| | | Tables | Columns | Tables | Columns | Tables | Columns | | | CTA | 44,769 | 116,887 | 456 | 1,769 | 609 | 1,851 | 82 | | CPA | 29,158 | 109,994 | 497 | 2,459 | 565 | 2,340 | 108 |

The WikiTables-TURL dataset was constructed by the authors of TURL and is based on the WikiTable corpus, which is a large collection of Wikipedia tables. The dataset consists of 580,171 tables divided into fixed training, validation and testing splits. Additionally, the dataset contains metadata about each table, such as the table name, table caption and column headers. 406,706 of these tables are annotated for the Column Type Annotation (CTA) task, 55,970 tables for the Columns Property Annotation (CPA) task and 200,744 tables for the Cell Entity Annotation (CEA) task. As classes for the CTA and CPA, Freebase's types and relations were used, whereas for the CEA task entities from Freebase were used. The table below lists the total annotated columns (or cells in the case of CEA) for each split and for each task as well as the number of classes used for annotation. | | Training | Validation | Testing | Classes | |-----|--------|----------|-------|-------| | CTA | 628,254 |13,391| 13,025 | 255 | | CPA | 62,954| 2,175 | 2,072 | 121 | | CEA | 1,264,217 | 76,720 | 225,777 | 1,787,737 | The authors have made the dataset and its variants publicly available for download.

1.0 Version of OpenEA benchmark datasets. Please use the updated 2.0 version, that has been subsequently released. Introduced by "A Benchmarking Study of Embedding-based Entity Alignment for Knowledge Graphs by Sun et. al, VLDB 2020" Contains entities from DBpedia, YAGO and Wikidata.

We present a further analysis of visual modality incompleteness, benchmarking latest MMEA models on our proposed dataset MMEA-UMVM. To create our MMEA-UMVM(uncertainly missing visual modality) datasets, we perform random image dropping on MMEA datasets. Specifically, we randomly discard entity images to achieve varying degrees of visual modality missing, ranging from 0.05 to the maximum $R_{img}$ of the raw datasets with a step of 0.05 or 0.1. Finally, we get a total number of 97 data split. Refer to the following paper for more details: Rethinking Uncertainly Missing and Ambiguous Visual Modality in Multi-Modal Entity Alignment

WDC Products is an entity matching benchmark which provides for the systematic evaluation of matching systems along combinations of three dimensions while relying on real-word data. The three dimensions are i) amount of corner-cases ii) generalization to unseen entities, and iii) development set size WDC Products contains 11715 product offers describing in total 2162 product entities belonging to various product categories. Source: WDC Products: A Multi-Dimensional Entity Matching Benchmark

DPB-5L is a Multilingual KG dataset containing 5 KGs in English, French, Japanese, Greek, and Spanish. The dataset is used for the Knowledge Graph Completion and Entity Alignment task. DPB-5L (Greek) is a subset of DPB-5L with Greek KG.

VizNet-Sato is a dataset from the authors of Sato and is based on the VizNet dataset. The authors choose from VizNet only relational web tables with headers matching their selected 78 DBpedia semantic types. The selected tables are divided into two categories: Full tables and Multi-column only tables. The first category corresponds to 78,733 selected tables from VizNet, while the second category includes 32,265 tables which have more than one column. The tables of both categories are divided into 5 subsets to be able to conduct 5-fold cross validation: 4 subsets are used for training and the last for evaluation. The headers of the columns act as semantic annotations for the Column Type Annotation (CTA) task. Some statistics about both categories of tables are provided in the table below, where "Columns" refers to the number of annotated columns and "Classes" to the number of unique DBpedia semantic types used for annotation. | | Columns | Classes | |-----------------|---------|---------| | Full | 120,609 | 78 | | Multi-column | 74,141 | 78 |

The WikipediaGS dataset was created by extracting Wikipedia tables from Wikipedia pages. It consists of 485,096 tables which were annotated with DBpedia entities for the Cell Entity Annotation (CEA) task. Additionally, a subset of these tables was annotated by Chen et al. for the Column Type Annotation (CTA) task and includes 604 tables, where selected columns were annotated using DBpedia types. This subset is available for download at their official Github repository. The table below shows the number of annotated cells/columns for each task and the number of different classes used for the annotation. | | Annotations | Classes | |-----|-------------|-----------| | CEA | 4,453,329 | 1,222,358 | | CTA | 620 | 31 |

A large-scale cross-lingual dataset for entity alignment

The GitTables-SemTab dataset is a subset of the GitTables dataset and was created to be used during the SemTab challenge. The dataset consists of 1101 tables and is used to benchmark the Column Type Annotation (CTA) task. Its columns were annotated using semantic properties from DBpedia and semantic types and properties from Schema.org. The table below shows the number of annotated columns and number of classes used to annotate this dataset. | | Columns | Classes | |----------------|-------|-------| | Column Type Annotation - Schema.org | 721 | 59 | | Column Type Annotation - DBpedia | 2,533 | 122 |

Click to add a brief description of the dataset (Markdown and LaTeX enabled). Provide: * a high-level explanation of the dataset characteristics * explain motivations and summary of its content * potential use cases of the dataset

The MusicBrainz20K dataset for entity resolution and entity clustering is based on real records about songs from the MusicBrainz database. Each record is described with the following attributes: artist, title, album, year and length. The records have been modified with the DAPO [1] data generator. The generated dataset consists of five sources and approximately 20K records describing 10K unique song entities. It contains duplicates for 50% of the original records in two to five sources which are generated with a high degree of corruption to stress-test the entity resolution and clustering approaches. [1] Hildebrandt, Kai, et al. "Large-scale data pollution with Apache Spark." IEEE Transactions on Big Data 6.2 (2017): 396-411.

We present TNCR, a new table dataset with varying image quality collected from free open source websites. TNCR dataset can be used for table detection in scanned document images and their classification into 5 different classes. TNCR contains 9428 high-quality labeled images. In this paper, we have implemented state-of-the-art deep learning-based methods for table detection to create several strong baselines. Cascade Mask R-CNN with ResNeXt-101-64x4d Backbone Network achieves the highest performance compared to other methods with a precision of 79.7%, recall of 89.8%, and f1 score of 84.4% on the TNCR dataset. We have made TNCR open source in the hope of encouraging more deep learning approaches to table detection, classification and structure recognition. Image source: https://github.com/abdoelsayed2016/TNCR_Dataset

WDC SOTAB is a benchmark that features two annotation tasks: Column Type Annotation and Columns Property Annotation. The goal of the Column Type Annotation (CTA) task is to annotate the columns of a table with 91 Schema.org types, such as telephone, duration, Place, or Organization. The goal of the Columns Property Annotation (CPA) task is to annotate pairs of table columns with one out of 176 Schema.org properties, such as gtin13, startDate, priceValidUntil, or recipeIngredient. The benchmark consists of 59,548 tables annotated for CTA and 48,379 tables annotated for CPA originating from 74,215 different websites. The tables are split into training-, validation- and test sets for both tasks. The tables cover 17 popular Schema.org types including Product, LocalBusiness, Event, and JobPosting. The tables originate from the Schema.org Table Corpus. Some characteristics for the different tasks are provided in the table below, where "Columns" refers to the number of columns/column pairs labeled and "Classes" to the number of unique classes used for annotation. | | Columns| Classes | |----------------|---------|---------| | Column Property Annotation | 174,998 | 176 | | Column Type Annotation | 162,351 | 91 |

Hand-disambiguation of a sample of U.S. patents inventor mentions from PatentsView.org. Inventors we selected indirectly by sampling inventor mentions uniformly at random. This results in inventor sampled with probability proportional to their number of granted patents. The time period considered is from 1976 to December 31, 2021, corresponding to the disambiguation labeled "disamb_inventor_id_20211230" in PatentsView's bulk data downloads "g_persistent_inventor.tsv" file (https://patentsview.org/download/data-download-tables). That is, the benchmark disambiguation intends to contain all inventor mentions for the sampled inventors from that time period. Note that the benchmark disambiguation contains a few extraneous mentions to patents granted outside of that time period. These should be ignored for evaluation purposes. The methodology used for the hand-disambiguation is described in Binette et al. (2022) (https://arxiv.org/abs/2210.01230). We used one disambiguation of 200 inventors from Binette et al. (2022), as well as an additional disambiguation of 200 inventors provided by an additional staff member. The two disambiguations were reviewed and validated. However, they should be expected to contain errors due to the ambiguous nature of inventor disambiguation. Furthermore, given the use as the December 30, 2021, disambiguation from PatentsView as a starting point of the hand-labeling, a bias towards this disambiguation should be expected.

CEREC is a large scale corpus for entity resolution in email conversations. The corpus consists of 6001 email threads from the Enron Email Corpus containing 36,448 email messages and 60,383 entity coreference chains. The annotation is carried out as a two-step process with minimal manual effort.

The DBP2.0 dataset can be downloaded from the figshare repository. It has three entity alignment settings, i.e., ZH-EN, JA-EN and FR-EN. Each setting has the following files: ent_links: reference entity alignment; rel_triples_1: relation triples in the ZH or JA or FR KG, list of triples like (h \t r \t t); rel_triples_2: relation triples in the EN KG; splits/train_links: training data for entity alignment, list of pairs like (e1 \t e2); splits/valid_links: validation data for entity alignment; splits/test_links: test data for entity alignment; splits/train_unlinked_ent1: training data for dangling entity detection, list of dangling entities in the ZH or JA or FR KG; splits/train_unlinked_ent2: training data for dangling entity detection, list of dangling entities in the EN KG; splits/valid_unlinked_ent1: validation data for dangling entity detection, list of dangling entities in the ZH or JA or FR KG; splits/valid_unlinked_ent2: validation data for dangling entity detection, list of dangling entities in the EN KG; splits/test_unlinked_ent1: test data for dangling entity detection, list of dangling entities in the ZH or JA or FR KG; splits/test_unlinked_ent2: test data for dangling entity detection, list of dangling entities in the EN KG; More information see: https://github.com/nju-websoft/OpenEA/tree/master/dbp2.0

Dataset for our paper Disambiguation-Centric Finetuning Makes Enterprise Tool-Calling LLMs More Realistic and Less Risky which includes 5000 enterprise tools and the corresponding dialogues generated using DiaFORGE UTC data engine. The dataset is generated with the data generation engine. The engine simulates a user agent and an assistant agent in a dialogue, where the user agent has a persona and the assistant agent has access to a set of tools. Detailed information about the data generation process can be found in the paper. Each entry in the dataset contains: - seed: Gold tool for the generated dialogue. - user_persona: Persona of the simulated user agent. - messages: List of messages in the dialogue, where each message is a dictionary containing: - distractor_tools: List of distractor tools that are not the gold tool but are relevant to the dialogue. These tools are used by user agent to generate utterances that are hard to disambiguate from the gold tool. - retrieved_tools: List of tools retrieved by the assistant agent and used by the assistant agent in order to ask clarifying questions to the user agent.

EventEA is an event-centric entity alignment dataset, harvested from EventKG, DBpedia and Wikidata. Source: EventEA: Benchmarking Entity Alignment for Event-centric Knowledge Graphs Image Source: https://arxiv.org/pdf/2211.02817v1.pdf

The dataset contains entities from IMDB, TheMovieDB and TheTVDB with goldstandard matches between the sources. Due to the licensing of IMDB we provide a script to build the IMDB part of the dataset yourself. The dataset contains a variety of entity types to match: persons, movies, series, episodes and companies.

PIZZA is a dataset for parsing pizza and drink orders, whose semantics cannot be captured by flat slots and intents. Source: PIZZA: A new benchmark for complex end-to-end task-oriented parsing Image Source: https://arxiv.org/pdf/2212.00265v1.pdf

WDC Block is a benchmark for comparing the performance of blocking methods that are used as part of entity resolution pipelines. Entity resolution aims to identify records in two datasets (A and B) that describe the same real-world entity. Since comparing all record pairs between two datasets can be computationally expensive, entity resolution is approached in two steps, blocking and matching. Blocking applies a computationally cheap method to remove non-matching record pairs and produces a smaller set of candidate record pairs reducing the workload of the matcher. During matching a more expensive pair-wise matcher produces a final set of matching record pairs. Existing benchmark datasets for blocking and matching are rather small with respect to the Cartesian product AxB for comparing all records and the vocabulary size. If blockers are evaluated only on these small datasets, effects resulting from a high number of records or from a large vocabulary size (large number of unique tokens that need to be indexed) may be missed. The Web Data Commons Block (WDC-Block) is a new blocking benchmark that provides much larger datasets and thus requires blockers that address these scalability challenges. WDC Block features a maximal Cartesian product of 200 billion pairs of product offers which were extracted form 3,259 e-shops. Additionally, we provide three development sets with different sizes (~1K pairs, ~5K pairs & ~20K pairs) to experiment with different amounts of training data for the blockers.

This dataset is used for user identity linkage across two online social networks in Chinese. It contains two popular Chinese social platforms: Sina Weibo\footnote{https://weibo.com} and Douban\footnote{https://www.douban.com}. Details: * 9,714 users and 117,218 relations in Weibo; 9,526 users and 120,245 relations in Douban; 1,397 pair of matched users. * Approximate power-law degree distribution and high aggregation coefficient. * Multiple text attributes available, including username, geographical location and recent (text) posts of the users. * Construction time: April 2020.