Column Type Annotation

GitTables is a corpus of currently 1M relational tables extracted from CSV files in GitHub covering 96 topics. Table columns in GitTables have been annotated with more than 2K different semantic types from Schema.org and DBpedia. The column annotations consist of semantic types, hierarchical relations, range types, table domain and descriptions. The tables were annotated using two methods: the semantic method and syntactic one. This leads to two kinds of annotations which in the metadata of the tables are referred to as syntactic and semantic annotations. The first method annotated 888,678 tables with Schema.org semantic types and 875,630 with DBpedia, while the second method annotated 1,161,117 tables with Schema.org and 1,156,601 with DBpedia semantic types. Some statistics about the tables are provided in the table below, "Columns" referring to the number of annotated columns and "Classes" to the number of unique DBpedia or Schema.org semantic types used for annotation. | | Columns | Classes | |----------------------|------------|---------| | Syntactic-DBpedia | 3,441,251 | 834 | | Syntactic-Schema.org | 2,671,588 | 677 | | Semantic-DBpedia | 10,757,184 | 2,380 | | Semantic-Schema.org | 10,475,155 | 2,407 |

The T2Dv2 dataset consists of 779 tables originating from the English-language subset of the WebTables corpus. 237 tables are annotated for the Table Type Detection task, 236 for the Columns Property Annotation (CPA) task and 235 for the Row Annotation task. The annotations that are used are DBpedia types, properties and entities. A subset of this dataset was annotated by Chen et al. for the Column Type Annotation (CTA) task where they annotate 236 tables with DBpedia types. In the papers it was used the subset was divided into training and testing splits and the evaluation was done on the testing split T2D-Te. This subset is available for download at their official Github repository. Some characteristics for the different tasks are provided in the table below, where "Annotations" refers to the number of cells/rows/columns/tables annotated and "Classes" to the number of unique classes used for annotation. | | Annotations | Classes | |----------------|---------|---------| | Column Property Annotation | 670 | 119 | | Row Annotation | 26,106 | 13,975 | | Table Type | 237 | 41 | | Column Type Annotation | 411 | 37 |

The ToughTables (2T) dataset was created for the SemTab challenge and includes 180 tables in total. The tables in this dataset can be categorized in two groups: the control (CTRL) group tables and tough (TOUGH) group tables. The CTRL group contains 60 tables generated by querying the DBpedia SPARQL endpoint and tables collected from Wikipedia and their characteristic is that they are easy to annotate. The TOUGH group contains 120 tables mainly scraped from the web, some containing misspelled words and nicknames/homonyms and their characteristic is that they are hard to annotate. In both groups some tables were generated by the authors where they added noise to the collected tables. The dataset was annotated for two tasks using DBpedia (DBP) types and entities and WikiData (WD): Column Type Annotation (CTA) and Cell Entity Annotation (CEA). In the table below the number of columns annotated for the CTA and number of cells annotated for the CEA task as well as the number of classes used are listed. | | Annotations| Classes | |-----|---------|---------| | DBP-Column Type Annotation | 540 | 39 | | DBP-Cell Entity Annotation | 663,656 | 16,023 | | WD-Column Type Annotation| 540 | 276 | | WD-Cell Entity Annotation | 667,244 | 24,653 |

The BioDiv dataset includes manually labeled tables for CTA and CEA from the biodiversity domain.

SOTAB V2 features two annotation tasks: Column Type Annotation (CTA) and Columns Property Annotation (CPA). The goal of the Column Type Annotation (CTA) task is to annotate the columns of a table using 82 types from the Schema.org vocabulary, such as telephone, Duration, Mass, or Organization. The goal of the Columns Property Annotation (CPA) task is to annotate pairs of table columns with one out of 108 Schema.org properties, such as gtin, startDate, priceValidUntil, or recipeIngredient. The benchmark consists of 45,834 tables annotated for CTA and 30,220 tables annotated for CPA originating from 55,511 different websites. The tables are split into training-, validation- and test sets for both tasks. The tables cover 17 popular Schema.org types including Product, LocalBusiness, Event, and JobPosting. Some characteristics for the different tasks are provided in the table below, where "Columns" refers to the number of columns/column pairs labeled and "Classes" to the number of unique classes used for annotation. | | Train | | Validation | | Test | | Classes | |-----|--------|---------|------------|---------|--------|---------|---------| | | Tables | Columns | Tables | Columns | Tables | Columns | | | CTA | 44,769 | 116,887 | 456 | 1,769 | 609 | 1,851 | 82 | | CPA | 29,158 | 109,994 | 497 | 2,459 | 565 | 2,340 | 108 |

The WikiTables-TURL dataset was constructed by the authors of TURL and is based on the WikiTable corpus, which is a large collection of Wikipedia tables. The dataset consists of 580,171 tables divided into fixed training, validation and testing splits. Additionally, the dataset contains metadata about each table, such as the table name, table caption and column headers. 406,706 of these tables are annotated for the Column Type Annotation (CTA) task, 55,970 tables for the Columns Property Annotation (CPA) task and 200,744 tables for the Cell Entity Annotation (CEA) task. As classes for the CTA and CPA, Freebase's types and relations were used, whereas for the CEA task entities from Freebase were used. The table below lists the total annotated columns (or cells in the case of CEA) for each split and for each task as well as the number of classes used for annotation. | | Training | Validation | Testing | Classes | |-----|--------|----------|-------|-------| | CTA | 628,254 |13,391| 13,025 | 255 | | CPA | 62,954| 2,175 | 2,072 | 121 | | CEA | 1,264,217 | 76,720 | 225,777 | 1,787,737 | The authors have made the dataset and its variants publicly available for download.

VizNet-Sato is a dataset from the authors of Sato and is based on the VizNet dataset. The authors choose from VizNet only relational web tables with headers matching their selected 78 DBpedia semantic types. The selected tables are divided into two categories: Full tables and Multi-column only tables. The first category corresponds to 78,733 selected tables from VizNet, while the second category includes 32,265 tables which have more than one column. The tables of both categories are divided into 5 subsets to be able to conduct 5-fold cross validation: 4 subsets are used for training and the last for evaluation. The headers of the columns act as semantic annotations for the Column Type Annotation (CTA) task. Some statistics about both categories of tables are provided in the table below, where "Columns" refers to the number of annotated columns and "Classes" to the number of unique DBpedia semantic types used for annotation. | | Columns | Classes | |-----------------|---------|---------| | Full | 120,609 | 78 | | Multi-column | 74,141 | 78 |

The WikipediaGS dataset was created by extracting Wikipedia tables from Wikipedia pages. It consists of 485,096 tables which were annotated with DBpedia entities for the Cell Entity Annotation (CEA) task. Additionally, a subset of these tables was annotated by Chen et al. for the Column Type Annotation (CTA) task and includes 604 tables, where selected columns were annotated using DBpedia types. This subset is available for download at their official Github repository. The table below shows the number of annotated cells/columns for each task and the number of different classes used for the annotation. | | Annotations | Classes | |-----|-------------|-----------| | CEA | 4,453,329 | 1,222,358 | | CTA | 620 | 31 |

The GitTables-SemTab dataset is a subset of the GitTables dataset and was created to be used during the SemTab challenge. The dataset consists of 1101 tables and is used to benchmark the Column Type Annotation (CTA) task. Its columns were annotated using semantic properties from DBpedia and semantic types and properties from Schema.org. The table below shows the number of annotated columns and number of classes used to annotate this dataset. | | Columns | Classes | |----------------|-------|-------| | Column Type Annotation - Schema.org | 721 | 59 | | Column Type Annotation - DBpedia | 2,533 | 122 |

WDC SOTAB is a benchmark that features two annotation tasks: Column Type Annotation and Columns Property Annotation. The goal of the Column Type Annotation (CTA) task is to annotate the columns of a table with 91 Schema.org types, such as telephone, duration, Place, or Organization. The goal of the Columns Property Annotation (CPA) task is to annotate pairs of table columns with one out of 176 Schema.org properties, such as gtin13, startDate, priceValidUntil, or recipeIngredient. The benchmark consists of 59,548 tables annotated for CTA and 48,379 tables annotated for CPA originating from 74,215 different websites. The tables are split into training-, validation- and test sets for both tasks. The tables cover 17 popular Schema.org types including Product, LocalBusiness, Event, and JobPosting. The tables originate from the Schema.org Table Corpus. Some characteristics for the different tasks are provided in the table below, where "Columns" refers to the number of columns/column pairs labeled and "Classes" to the number of unique classes used for annotation. | | Columns| Classes | |----------------|---------|---------| | Column Property Annotation | 174,998 | 176 | | Column Type Annotation | 162,351 | 91 |