Papers with Code 아카이브

Relation Extraction

태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.

하위 태스크
15
평가표
51
보존된 평가 기록
373

Relation Extraction is the task of predicting attributes and relations for entities in a sentence. For example, given a sentence “Barack Obama was born in Honolulu, Hawaii.”, a relation classifier aims at predicting the relation of “bornInCity”. Relation Extraction is the key component for building relation knowledge graphs, and it is of crucial significance to natural language processing applications such as structured search, sentiment analysis, question answering, and summarization.

Source: Deep Residual Learning for Weakly-Supervised Relation Extraction

데이터셋

출처에 같은 태스크 이름으로 연결된 데이터셋입니다. 동명 태스크가 포함될 수 있으므로, 비교할 때는 평가 결과의 조건도 확인하세요.

GUM

GUM is an open source multilayer English corpus of richly annotated texts from twelve text types. Annotations include: * Multiple POS tags, morphological features and lemmatization * Sentence segmentation and rough speech act * Document structure in TEI XML (paragraphs, headings, figures, etc.) * ISO date/time annotations * Speaker and addressee information (where relevant) * Constituent and dependency syntax * Information status (given, accessible, new, split antecedent) * Entity and coreference annotation, including bridging anaphora * Entity linking (Wikification) * Discourse parses in Rhetorical Structure Theory and discourse dependencies

CDCP

The Cornell eRulemaking Corpus – CDCP is an argument mining corpus annotated with argumentative structure information capturing the evaluability of arguments. The corpus consists of 731 user comments on Consumer Debt Collection Practices (CDCP) rule by the Consumer Financial Protection Bureau (CFPB); the resulting dataset contains 4931 elementary unit and 1221 support relation annotations. It is a resource for building argument mining systems that can not only extract arguments from unstructured text, but also identify what additional information is necessary for readers to understand and evaluate a given argument. Immediate applications include providing real-time feedback to commenters, specifying which types of support for which propositions can be added to construct better-formed arguments.

CDR

The BioCreative V CDR task corpus is manually annotated for chemicals, diseases and chemical-induced disease (CID) relations. It contains the titles and abstracts of 1500 PubMed articles and is split into equally sized train, validation and test sets. It is common to first tune a model on the validation set and then train on the combination of the train and validation sets before evaluating on the test set. It is also common to filter negative relations with disease entities that are hypernyms of a corresponding true relations disease entity within the same abstract (see Appendix C of this paper for details).

TekGen

The Dataset is part of the KELM corpus This is the Wikipedia text--Wikidata KG aligned corpus used to train the data-to-text generation model. Please note that this is a corpus generated with distant supervision and should not be used as gold standard for evaluation. It consists of 3 files: https://storage.googleapis.com/gresearch/kelm-corpus/updated-2021/quadruples-train.tsv https://storage.googleapis.com/gresearch/kelm-corpus/updated-2021/quadruples-validation.tsv https://storage.googleapis.com/gresearch/kelm-corpus/updated-2021/quadruples-test.tsv Each file contains one example per line. Each example is a json object with three fields: triples: A list of triples of the form (subject, relation, object). eg. (Person X, award received, Award Y). If the triple has a subproperty, then it is quadruple instead. eg. (Person X, Award Y, received on, Date Z). serialized triples: triples concatenated together as used for input to T5. The format is "<subject> <relation> <object>" where some subjects have multiple relations, e.g. "<subject> <relation1> <object1> <relation2> <object2> <relation3> <object3>". For more details on how these relations are grouped, please refer to the paper. sentence: The wikipedia sentence aligned to these triples. The names, aliases and Wikidata Ids of the entities can be found in https://storage.googleapis.com/gresearch/kelm-corpus/updated-2021/entities.jsonl.

Discovery

The Discovery datasets consists of adjacent sentence pairs (s1,s2) with a discourse marker (y) that occurred at the beginning of s2. They were extracted from the depcc web corpus. Markers prediction can be used in order to train a sentence encoders. Discourse markers can be considered as noisy labels for various semantic tasks, such as entailment (y=therefore), subjectivity analysis (y=personally) or sentiment analysis (y=sadly), similarity (y=similarly), typicality, (y=curiously) ... The specificity of this dataset is the diversity of the markers, since previously used data used only ~10 imbalanced classes. The author of the dataset provide: - a list of the 174 discourse markers - a Base version of the dataset with 1.74 million pairs (10k examples per marker) - a Big version with 3.4 million pairs - a Hard version with 1.74 million pairs where the connective couldn't be predicted with a fastText linear model Source: GitHub

GDA

The gene-disease associations corpus contains 30,192 titles and abstracts from PubMed articles that have been automatically labelled for genes, diseases and gene-disease associations via distant supervision. The test set is comprised of 1000 of these examples. It is common to hold out a random 20% of the examples in the train set as a validation set.

REFinD

REFinD is a large-scale annotated dataset of relations, with ∼29K instances and 22 relations amongst 8 types of entity pairs, generated entirely over financial documents.

2012 i2b2 Temporal Relations

The Sixth Informatics for Integrating Biology and the Bedside (i2b2) Natural Language Processing Challenge for Clinical Records focused on the temporal relations in clinical narratives. The organizers provided the research community with a corpus of discharge summaries annotated with temporal information, to be used for the development and evaluation of temporal reasoning systems. 18 teams from around the world participated in the challenge. During the workshop, participating teams presented comprehensive reviews and analysis of their systems, and outlined future research directions suggested by the challenge contributions.

FinRED

FinRED is a relation extraction dataset curated from financial news and earning call transcripts containing relations from the finance domain. FinRED has been created by mapping Wikidata triplets using distance supervision method.

2018 n2c2 (Track 2) - Adverse Drug Events and Medication Extraction

Abstract Objective This article summarizes the preparation, organization, evaluation, and results of Track 2 of the 2018 National NLP Clinical Challenges shared task. Track 2 focused on extraction of adverse drug events (ADEs) from clinical records and evaluated 3 tasks: concept extraction, relation classification, and end-to-end systems. We perform an analysis of the results to identify the state of the art in these tasks, learn from it, and build on it. Materials and Methods For all tasks, teams were given raw text of narrative discharge summaries, and in all the tasks, participants proposed deep learning–based methods with hand-designed features. In the concept extraction task, participants used sequence labelling models (bidirectional long short-term memory being the most popular), whereas in the relation classification task, they also experimented with instance-based classifiers (namely support vector machines and rules). Ensemble methods were also popular. Results A total of 28 teams participated in task 1, with 21 teams in tasks 2 and 3. The best performing systems set a high performance bar with F1 scores of 0.9418 for concept extraction, 0.9630 for relation classification, and 0.8905 for end-to-end. However, the results were much lower for concepts and relations of Reasons and ADEs. These were often missed because local context is insufficient to identify them. Conclusions This challenge shows that clinical concept extraction and relation classification systems have a high performance for many concept types, but significant improvement is still required for ADEs and Reasons. Incorporating the larger context or outside knowledge will likely improve the performance of future systems.

BB

The Bacteria Biotope (BB) Task is part of the BioNLP Open Shared Tasks and meets the BioNLP-OST standards of quality, originality and data formats. Manually annotated data is provided for training, development and evaluation of information extraction methods. Tools for the detailed evaluation of system outputs are available. Support in performing linguistic processing are provided in the form of analyses created by various state-of-the art tools on the dataset texts. ##Motivation Biology and bioinformatics projects produce huge amounts of heterogeneous information about the microbial strains that have been experimentally identified in a given environment (habitat), and theirs properties (phenotype). These projects include applied microbiology domain (food safety), health sciences and waste processing. Knowledge about microbial diversity is critical for studying in depth the microbiome, the interaction mechanisms of bacteria with their environment from genetic, phylogenetic and ecology perspectives. A large part of the information is expressed in free text in large sets of scientific papers, web pages or databases. Thus, automatic systems are needed to extract the relevant information. The BB task aims to encourage the development of such systems. ##BB Task Goal The BB Task is an information extraction task involving entity recognition, entity normalization and relation extraction. The BB Task consists in recognizing mentions of microorganisms and microbial biotopes and phenotypes in scientific and textbook text, normalizing these mentions according to domain knowledge resources (a taxonomy and an ontology), and extracting relations between them. It is the new edition of the Bacteria Biotope task previously run at BioNLP Shared Task 2016, 2013 and 2011. The task has been extended to include new entity and relation types and new documents.

HyperRED

HyperRED is a dataset for the new task of hyper-relational extraction, which extracts relation triplets together with qualifier information such as time, quantity or location. For example, the relation triplet (Leonard Parker, Educated At, Harvard University) can be factually enriched by including the qualifier (End Time, 1967). HyperRED contains 44k sentences with 62 relation types and 44 qualifier types.

TACRED-Revisited

The TACRED-Revisited dataset improves the crowd-sourced TACRED dataset for relation extraction by relabeling the dev and test sets using expert linguistic annotators. Relabeling focuses on the 5K most challenging instances in dev and test, in total, 51.2% of these are corrected. Published at ACL 2020. Paper (arXiv): https://arxiv.org/abs/2004.14855

French Timebank

French TimeBank, a corpus for French annotated in ISO-TimeML. Some statistics: - Documents: 109 - Events: 2100 - Timexs: 608 - Signal: 288 - ALink: 36 - SLink: 457 - TLinks: 191

KnowledgeNet

KnowledgeNet is a benchmark dataset for the task of automatically populating a knowledge base (Wikidata) with facts expressed in natural language text on the web. KnowledgeNet provides text exhaustively annotated with facts, thus enabling the holistic end-to-end evaluation of knowledge base population systems as a whole, unlike previous benchmarks that are more suitable for the evaluation of individual subcomponents (e.g., entity linking, relation extraction). For instance, the dataset contains text expressing the fact (Gennaro Basile; RESIDENCE; Moravia), in the passage: "Gennaro Basile was an Italian painter, born in Naples but active in the German-speaking countries. He settled at Brünn, in Moravia, and lived about 1756..." Source: KnowledgeNet

PGR

Phenotype-Gene Relations (PGR) is a corpus that consists of 1712 abstracts, 5676 human phenotype annotations, 13835 gene annotations, and 4283 relations. Source: A Silver Standard Corpus of Human Phenotype-Gene Relations

WNUT 2020

The training and development dataset for our task was taken from previous work on wet lab corpus (Kulkarni et al., 2018) that consists of from the 623 protocols. We excluded the eight duplicate protocols from this dataset and then re-annotated the 615 unique protocols in BRAT (Stenetorp et al., 2012).

GAD

GAD, or Gene Associations Database, is a corpus of gene-disease associations curated from genetic association studies.

LabPics

LabPics Chemistry Dataset Dataset for computer vision for materials segmentation and classification in chemistry labs, medical labs, and any setting where materials are handled inside containers. The Vector-LabPics dataset comprises 7900 images of materials in various phases and processes within mostly transparent vessels in chemistry labs, medical labs and hospitals, and other environments. The images are annotated for both the vessels and the individual material phases inside them, and each instance is assigned one or more classes (liquid, solid, foam, suspension, powder, gel, granular, vapor) . The fill level, labels, corks, and other parts of the vessel are also annotated. The material classes cover the main states of matter, including liquids, solids, vapors, foams, gels, and subcategories like powder, granular, and suspension. Relationships between materials, such as which material is immersed inside other materials, are annotated. The vessel class cover glassware, labware plates, bottles, and any other type of vessel that is used to contain or carry materials. The type of vessel (e.g., syringe, tube, cup, infusion bottle/bag), and the properties of the vessel (transparent, opaque) are annotated. In addition, vessel parts such as corks, labels, spikes, and valves are annotated. Relations and hierarchies between vessels and materials are also annotated, such as which vessel contains which material or which vessels are linked or contain each other. The images were collected from various contributors and covered most aspects of chemistry lab works as well as a variety of other fields where materials are handled in container vessels. Documents specifying annotation formats are available inside the dataset file. Version 1 contain 2200 images with simple instance and semantic annotations, and is relatively simple to use, it is described in the paper "Computer Vision for Recognition of Materials and Vessels in Chemistry Lab Settings and the Vector-LabPics Data Set" Format The dataset contains annotated images for both material and vessels in chemistry labs, medical labs, and any area where liquids and solids are handled within vessels. There are two levels of annotation for each image. One annotation set for vessels and the second for the material phases inside these vessels. Vessels are defined as any container that can carry materials such as Jars, Erlenmayers, Tubes, Funnels, syringes, IV bags, and any other labware or glassware that can contain or carry materials. Material phases are any material contained within or on the vessel. For example, for two-phase separating liquids, each liquid phase is annotated as one instance. If there is foam above the liquid or a chunk of solid inside the liquid, the foam, liquid, and solid will be annotated as different phases. In addition, vessel parts like cork, labels, and valves are annotated as instances. For each instance, there is a list of all the classes it belongs to, and a list of its property. For vessels, the instance classes are the vessel type (Cup, jar, Separatory-funnel…) and the vessel properties (Transparent, Opaque…). For materials, the classes are the material types ( Liquid, solid, suspension, foam, powder…) and their properties (Scattered, On vessel surface…), and for parts, the part type (cork/label). In addition, the relations between instances are annotated. This includes which material instances are inside which vessels, which vessels are linked to each other or are inside each other (for vessels inside other vessels), and which material phase is immersed inside another material phase. In addition to instance segmentation maps, the dataset also includes semantic segmentation maps that give each pixel in the image all the classes to which it belongs. In other words, for each class (Liquid, Solid, Vessel, Foam), there is a map of all the regions in the image belonging to this class. Note that every pixel and every instance can have several classes. In addition, instances often overlap, like in the case of material inside the vessel, vessel inside the vessel, and material phase immersed inside other material (like solid inside liquid).

SoMeSci

Knowledge about software used in scientific investigations is important for several reasons, for instance, to enable an understanding of provenance and methods involved in data handling. However, software is usually not formally cited, but rather mentioned informally within the scholarly description of the investigation, raising the need for automatic information extraction and disambiguation. Given the lack of reliable ground truth data, we present SoMeSci - Software Mentions in Science - a gold standard knowledge graph of software mentions in scientific articles. It contains high quality annotations (IRR: κ = .82) of 3756 software mentions in 1367 PubMed Central articles. Besides the plain mention of the software, we also provide relation labels for additional information, such as the version, the developer, a URL or citations. Moreover, we distinguish between different types, such as application, plugin or programming environment, as well as different types of mentions, such as usage or creation. To the best of our knowledge, SoMeSci is the most comprehensive corpus about software mentions in scientific articles, providing training samples for Named Entity Recognition, Relation Extraction, Entity Disambiguation, and Entity Linking. Finally, we sketch potential use cases and provide baseline results for the different tasks.

ChemDisGene

ChemDisGene, a new dataset for training and evaluating multi-class multi-label biomedical relation extraction models.

DDRel

DDRel is a dataset for interpersonal relation classification in dyadic dialogues. It consists of 6,300 dyadic dialogue sessions between 694 pairs of speakers with 53,126 utterances in total. It is constructed by crawling movie scripts from IMSDb and annotating the relation labels for each session according to 13 pre-defines relationships. Source: https://github.com/JiaQiSJTU/DialogueRelationClassification

DiS-ReX

DiS-ReX is a multilingual dataset for distantly supervised (DS) relation extraction (RE). The dataset has over 1.5 million instances, spanning 4 languages (English, Spanish, German and French). The dataset has 36 positive relation types + 1 no relation (NA) class.

NYT10-HRL

a dataset from A Hierarchical Framework for Relation Extraction with Reinforcement Learning

Perlex

Persian dataset for relation extraction, which is an expert-translated version of the "Semeval-2010-Task-8" dataset. Source: PERLEX: A Bilingual Persian-English Gold Dataset for Relation Extraction

SemEval-2022 Task-12

Symlink is a SemEval shared task of extracting mathematical symbols and their descriptions from LaTeX source of scientific documents. This is a new task in SemEval 2022, which attracted 180 individual registrations and 59 final submissions from 7 participant teams.

TimeBankPT

TimeBankPT is a corpus of Portuguese text with annotations about time. The annotation scheme used is similar to TimeML. TimeBankPT is the result of adapting the English corpus used in the first TempEval challenge to the Portuguese language. In what regards the temporal relaiton type, the corpus was annotated with six labels, namely: BEFORE, AFTER, OVERLAP, BEFORE-OR-OVERLAP, OVERLAP-OR-AFTER and VAGUE. | | Train | Test | |-----------|-------|------| | Documents | 162 | 20 | | Sentences | 2281 | 351 | | Words | 60782 | 8920 | | Events | 6790 | 1097 | | Timexs | 1244 | 165 | | Tlinks | 5781 | 758 |

DISRPT2021

The DISRPT 2021 shared task, co-located with CODI 2021 at EMNLP, introduces the second iteration of a cross-formalism shared task on discourse unit segmentation and connective detection, as well as the first iteration of a cross-formalism discourse relation classification task. We provide training, development and test datasets from all available languages and treebanks in the RST, SDRT and PDTB formalisms, using a uniform format. Because different corpora, languages and frameworks use different guidelines, the shared task is meant to promote design of flexible methods for dealing with various guidelines, and help to push forward the discussion of standards for computational approaches to discourse relations. We include data for evaluation with and without gold syntax, or otherwise using provided automatic parses for comparison to gold syntax data.

FOBIE

The Focused Open Biology Information Extraction (FOBIE) dataset aims to support IE from Computer-Aided Biomimetics. The dataset contains ~1,500 sentences from scientific biological texts. These sentences are annotated with TRADE-OFFS and syntactically similar relations between unbounded arguments, as well as argument-modifiers. The FOBIE dataset has been used to explore Semi-Open Relation Extraction (SORE). The code for this and instructions can be found inside the SORE folder Readme.md, or in the ReadTheDocs documentations. Source: FOBIE

FREDo

FREDo is a Few-Shot Document-Level Relation Extraction Benchmark based on DocRED and SciERC. The dataset is divided into four subsets: training set (62 relations), validation set (16 relations), in-domain test set (16 relations), and cross-domain test set (7 relations).

ROOR

ROOR is a reading order prediction (ROP) benchmark which annotates layout reading order as ordering relations. Layout reading order is typically formulated as a permutation of layout elements, i.e. a sequence containing all the layout elements. However, multiple cases have reflected that this formulation does not adequately convey the complete reading order information in the layout, which may negatively affect the utilization of this signal in downstream VrD tasks. Therefore, this work investigate the properties of layout reading order, conceptualizing it with terms Immediate Succession During Reading(ISDR) and Generalized Succession During Reading(GSDR), and formulate each of which as an ordering relation over layout elements. Then, ROOR provides the annotation of the ISDR relationship over layout segments, based on the layout annotation of EC-FUNSD. Overall, ROOR comprises 199 samples including 10,662 segments, 31,297 words and 10,967 annotated reading order linking pairs. We hope the construction of this benchmark could facilitate the development of automated ROP methods of the improved task form.

SupplyGraph

Graph Neural Networks (GNNs) have gained traction across different domains such as transportation, bio-informatics, language processing, and computer vision. However, there is a noticeable absence of research on applying GNNs to supply chain networks. Supply chain networks are inherently graphlike in structure, making them prime candidates for applying GNN methodologies. This opens up a world of possibilities for optimizing, predicting, and solving even the most complex supply chain problems. A major setback in this approach lies in the absence of real-world benchmark datasets to facilitate the research and resolution of supply chain problem using GNNs. To address the issue, we present a real-world benchmark dataset for temporal tasks, obtained from one of the leading FMCG companies in Bangladesh, focusing on supply chain planning for production purposes. The dataset includes temporal data as node features to enable sales predictions, production planning, and the identification of factory issues. By utilizing this dataset, researchers can employ GNNs to address numerous supply chain problems, thereby advancing the field of supply chain analytics and planning. - Dataset GitHub - arXiv - PDF on arXiv Read the paper to learn more details and data statistics.

출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.