Papers with Code 아카이브

Relation Classification

태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.

하위 태스크
3
평가표
8
보존된 평가 기록
33

Relation Classification is the task of identifying the semantic relation holding between two nominal entities in text.

Source: Structure Regularized Neural Network for Entity Relation Classification for Chinese Literature Text

데이터셋

출처에 같은 태스크 이름으로 연결된 데이터셋입니다. 동명 태스크가 포함될 수 있으므로, 비교할 때는 평가 결과의 조건도 확인하세요.

TACRED

TACRED is a large-scale relation extraction dataset with 106,264 examples built over newswire and web text from the corpus used in the yearly TAC Knowledge Base Population (TAC KBP) challenges. Examples in TACRED cover 41 relation types as used in the TAC KBP challenges (e.g., per:schools_attended and org:members) or are labeled as no_relation if no defined relation is held. These examples are created by combining available human annotations from the TAC KBP challenges and crowdsourcing. Source: https://nlp.stanford.edu/projects/tacred/

DocRED

DocRED (Document-Level Relation Extraction Dataset) is a relation extraction dataset constructed from Wikipedia and Wikidata. Each document in the dataset is human-annotated with named entity mentions, coreference information, intra- and inter-sentence relations, and supporting evidence. DocRED requires reading multiple sentences in a document to extract entities and infer their relations by synthesizing all information of the document. Along with the human-annotated data, the dataset provides large-scale distantly supervised data. DocRED contains 132,375 entities and 56,354 relational facts annotated on 5,053 Wikipedia documents. In addition to the human-annotated data, the dataset provides large-scale distantly supervised data over 101,873 documents. Source: DocRED: A Large-Scale Document-Level Relation Extraction Dataset Image Source: DocRED: A Large-Scale Document-Level Relation Extraction Dataset

SciERC

SciERC dataset is a collection of 500 scientific abstract annotated with scientific entities, their relations, and coreference clusters. The abstracts are taken from 12 AI conference/workshop proceedings in four AI communities, from the Semantic Scholar Corpus. SciERC extends previous datasets in scientific articles SemEval 2017 Task 10 and SemEval 2018 Task 7 by extending entity types, relation types, relation coverage, and adding cross-sentence relations using coreference links. Source: http://nlp.cs.washington.edu/sciIE/ Image Source: http://nlp.cs.washington.edu/sciIE/

FewRel 2.0

A more challenging task to investigate two aspects of few-shot relation classification models: (1) Can they adapt to a new domain with only a handful of instances? (2) Can they detect none-of-the-above (NOTA) relations? Source: FewRel 2.0: Towards More Challenging Few-Shot Relation Classification

MATRES

This is the Multi-Axis Temporal RElations for Start-points (i.e., MATRES) dataset

GUM

GUM is an open source multilayer English corpus of richly annotated texts from twelve text types. Annotations include: * Multiple POS tags, morphological features and lemmatization * Sentence segmentation and rough speech act * Document structure in TEI XML (paragraphs, headings, figures, etc.) * ISO date/time annotations * Speaker and addressee information (where relevant) * Constituent and dependency syntax * Information status (given, accessible, new, split antecedent) * Entity and coreference annotation, including bridging anaphora * Entity linking (Wikification) * Discourse parses in Rhetorical Structure Theory and discourse dependencies

CDCP

The Cornell eRulemaking Corpus – CDCP is an argument mining corpus annotated with argumentative structure information capturing the evaluability of arguments. The corpus consists of 731 user comments on Consumer Debt Collection Practices (CDCP) rule by the Consumer Financial Protection Bureau (CFPB); the resulting dataset contains 4931 elementary unit and 1221 support relation annotations. It is a resource for building argument mining systems that can not only extract arguments from unstructured text, but also identify what additional information is necessary for readers to understand and evaluate a given argument. Immediate applications include providing real-time feedback to commenters, specifying which types of support for which propositions can be added to construct better-formed arguments.

Discovery

The Discovery datasets consists of adjacent sentence pairs (s1,s2) with a discourse marker (y) that occurred at the beginning of s2. They were extracted from the depcc web corpus. Markers prediction can be used in order to train a sentence encoders. Discourse markers can be considered as noisy labels for various semantic tasks, such as entailment (y=therefore), subjectivity analysis (y=personally) or sentiment analysis (y=sadly), similarity (y=similarly), typicality, (y=curiously) ... The specificity of this dataset is the diversity of the markers, since previously used data used only ~10 imbalanced classes. The author of the dataset provide: - a list of the 174 discourse markers - a Base version of the dataset with 1.74 million pairs (10k examples per marker) - a Big version with 3.4 million pairs - a Hard version with 1.74 million pairs where the connective couldn't be predicted with a fastText linear model Source: GitHub

TACRED-Revisited

The TACRED-Revisited dataset improves the crowd-sourced TACRED dataset for relation extraction by relabeling the dev and test sets using expert linguistic annotators. Relabeling focuses on the 5K most challenging instances in dev and test, in total, 51.2% of these are corrected. Published at ACL 2020. Paper (arXiv): https://arxiv.org/abs/2004.14855

LabPics

LabPics Chemistry Dataset Dataset for computer vision for materials segmentation and classification in chemistry labs, medical labs, and any setting where materials are handled inside containers. The Vector-LabPics dataset comprises 7900 images of materials in various phases and processes within mostly transparent vessels in chemistry labs, medical labs and hospitals, and other environments. The images are annotated for both the vessels and the individual material phases inside them, and each instance is assigned one or more classes (liquid, solid, foam, suspension, powder, gel, granular, vapor) . The fill level, labels, corks, and other parts of the vessel are also annotated. The material classes cover the main states of matter, including liquids, solids, vapors, foams, gels, and subcategories like powder, granular, and suspension. Relationships between materials, such as which material is immersed inside other materials, are annotated. The vessel class cover glassware, labware plates, bottles, and any other type of vessel that is used to contain or carry materials. The type of vessel (e.g., syringe, tube, cup, infusion bottle/bag), and the properties of the vessel (transparent, opaque) are annotated. In addition, vessel parts such as corks, labels, spikes, and valves are annotated. Relations and hierarchies between vessels and materials are also annotated, such as which vessel contains which material or which vessels are linked or contain each other. The images were collected from various contributors and covered most aspects of chemistry lab works as well as a variety of other fields where materials are handled in container vessels. Documents specifying annotation formats are available inside the dataset file. Version 1 contain 2200 images with simple instance and semantic annotations, and is relatively simple to use, it is described in the paper "Computer Vision for Recognition of Materials and Vessels in Chemistry Lab Settings and the Vector-LabPics Data Set" Format The dataset contains annotated images for both material and vessels in chemistry labs, medical labs, and any area where liquids and solids are handled within vessels. There are two levels of annotation for each image. One annotation set for vessels and the second for the material phases inside these vessels. Vessels are defined as any container that can carry materials such as Jars, Erlenmayers, Tubes, Funnels, syringes, IV bags, and any other labware or glassware that can contain or carry materials. Material phases are any material contained within or on the vessel. For example, for two-phase separating liquids, each liquid phase is annotated as one instance. If there is foam above the liquid or a chunk of solid inside the liquid, the foam, liquid, and solid will be annotated as different phases. In addition, vessel parts like cork, labels, and valves are annotated as instances. For each instance, there is a list of all the classes it belongs to, and a list of its property. For vessels, the instance classes are the vessel type (Cup, jar, Separatory-funnel…) and the vessel properties (Transparent, Opaque…). For materials, the classes are the material types ( Liquid, solid, suspension, foam, powder…) and their properties (Scattered, On vessel surface…), and for parts, the part type (cork/label). In addition, the relations between instances are annotated. This includes which material instances are inside which vessels, which vessels are linked to each other or are inside each other (for vessels inside other vessels), and which material phase is immersed inside another material phase. In addition to instance segmentation maps, the dataset also includes semantic segmentation maps that give each pixel in the image all the classes to which it belongs. In other words, for each class (Liquid, Solid, Vessel, Foam), there is a map of all the regions in the image belonging to this class. Note that every pixel and every instance can have several classes. In addition, instances often overlap, like in the case of material inside the vessel, vessel inside the vessel, and material phase immersed inside other material (like solid inside liquid).

DISRPT2021

The DISRPT 2021 shared task, co-located with CODI 2021 at EMNLP, introduces the second iteration of a cross-formalism shared task on discourse unit segmentation and connective detection, as well as the first iteration of a cross-formalism discourse relation classification task. We provide training, development and test datasets from all available languages and treebanks in the RST, SDRT and PDTB formalisms, using a uniform format. Because different corpora, languages and frameworks use different guidelines, the shared task is meant to promote design of flexible methods for dealing with various guidelines, and help to push forward the discussion of standards for computational approaches to discourse relations. We include data for evaluation with and without gold syntax, or otherwise using provided automatic parses for comparison to gold syntax data.

FREDo

FREDo is a Few-Shot Document-Level Relation Extraction Benchmark based on DocRED and SciERC. The dataset is divided into four subsets: training set (62 relations), validation set (16 relations), in-domain test set (16 relations), and cross-domain test set (7 relations).

SupplyGraph

Graph Neural Networks (GNNs) have gained traction across different domains such as transportation, bio-informatics, language processing, and computer vision. However, there is a noticeable absence of research on applying GNNs to supply chain networks. Supply chain networks are inherently graphlike in structure, making them prime candidates for applying GNN methodologies. This opens up a world of possibilities for optimizing, predicting, and solving even the most complex supply chain problems. A major setback in this approach lies in the absence of real-world benchmark datasets to facilitate the research and resolution of supply chain problem using GNNs. To address the issue, we present a real-world benchmark dataset for temporal tasks, obtained from one of the leading FMCG companies in Bangladesh, focusing on supply chain planning for production purposes. The dataset includes temporal data as node features to enable sales predictions, production planning, and the identification of factory issues. By utilizing this dataset, researchers can employ GNNs to address numerous supply chain problems, thereby advancing the field of supply chain analytics and planning. - Dataset GitHub - arXiv - PDF on arXiv Read the paper to learn more details and data statistics.

CORE

Click to add a brief description of the dataset (Markdown and LaTeX enabled). Provide: * a high-level explanation of the dataset characteristics * explain motivations and summary of its content * potential use cases of the dataset

DRI Corpus

The Dr. Inventor Multi-Layer Scientific Corpus (DRI Corpus) includes 40 Computer Graphics papers, selected by domain experts. Each paper of the Corpus has been annotated by three annotators by providing the following layers of annotations, each one characterizing a core aspect of scientific publications: * Scientific discourse: each sentence has been associated to a specific scientific discourse category (Background, Approach, Challenge, Future Work, etc.). * Subjective statements and novelty: each sentence has been characterized with respect to advantages, disadvantages and novel aspects presented. * Citation purpose: to each citation has been associated a purpose specifying the reason why the authors of the paper cited the specific piece of research. * Summary relevance of sentences and hand written summaries: each sentence of the paper has been characterized by an integer score ranging from 1 to 5, to point out the relevance of the same sentence for its inclusion in the summary of the paper. Sentences rated as 5 are the most relevant ones to summarize a paper. For each paper three hand-written summaries (max 250 words) are provided. Source: Dr. Inventor Multi-layer Scientific Corpus

MultiTACRED

MultiTACRED is a multilingual version of the large-scale TAC Relation Extraction Dataset. It covers 12 typologically diverse languages from 9 language families, and was created by the Speech & Language Technology group of DFKI by machine-translating the instances of the original TACRED dataset and automatically projecting their entity annotations. For details of the original TACRED's data collection and annotation process, see the Stanford paper. Translations are syntactically validated by checking the correctness of the XML tag markup. Any translations with an invalid tag structure, e.g. missing or invalid head or tail tag pairs, are discarded (on average, 2.3% of the instances). Languages covered are: Arabic, Chinese, Finnish, French, German, Hindi, Hungarian, Japanese, Polish, Russian, Spanish, Turkish. Intended use is supervised relation classification. Audience - researchers. MultiTACRED is released via the Linguistic Data Consortium (LDC License). You can download MultiTACRED from the LDC MultiTACRED webpage. Please see our ACL paper for full details.

TexRel

Green family of datasets for emergent communications on relations. By comparison with other relations datasets, TexRel provides rapid training and experimentation, whilst being sufficiently large to avoid overfitting in the context of emergent communications.

Translated TACRED

533 parallel examples sampled from TACRED, translated into Russian and Korean (and 3 additional examples in Russian), accompanied with tranlsation of a list of trigger words collected for the different relations.

TFH_Annotated_Dataset

Dataset Introduction TFH_Annotated_Dataset is an annotated patent dataset pertaining to thin film head technology in hard-disk. To the best of our knowledge, this is the second labeled patent dataset public available in technology management domain that annotates both entities and the semantic relations between entities, the first one is [1]. The well-crafted information schema used for patent annotation contains 17 types of entities and 15 types of semantic relations as shown below. Table 1 The specification of entity types | Type | Comment | example | | ------------------ | ------------------------------------------- | :----------------------------------------------------------- | | physical flow | substance that flows freely | The etchant solution has a suitable solvent additive such as glycerol or methyl cellulose | | information flow | information data | A camera using a film having a magnetic surface for recording magnetic data thereon | | energy flow | entity relevant to energy | Conductor is utilized for producing writing flux in magnetic yoke | | measurement | method of measuring something | The curing step takes place at the substrate temperature less than 200.degree | | value | numerical amount | The curing step takes place at the substrate temperature less than 200.degree | | location | place or position | The legs are thinner near the pole tip than in the back gap region | | state | particular condition at a specific time | The MR elements are biased to operate in a magnetically unsaturated mode | | effect | change caused an innovation | Magnetic disk system permits accurate alignment of magnetic head with spaced tracks | | function | manufacturing technique or activity | A magnetic head having highly efficient write and read functions is thereby obtained | | shape | the external form or outline of something | Recess is filled with non-magnetic material such as glass | | component | a part or element of a machine | A pole face of yoke is adjacent edge of element remote from surface | | attribution | a quality or feature of something | A pole face of yoke is adjacent edge of element remote from surface | | consequence | The result caused by something or activity | This prevents the slider substrate from electrostatic damage | | system | a set of things working together as a whole | A digital recording system utilizing a magnetoresistive transducer in a magnetic recording head | | material | the matter from which a thing is made | Interlayer may comprise material such as Ta | | scientific concept | terminology used in scientific theory | Peak intensity ratio represents an amount hydrophilic radical | | other | Not belongs to the above entity types | Pressure distribution across air bearing surface is substantially symmetrical side | Table 2 The specification of relation types | TYPE | COMMENT | EXAMPLE | | ------------------ | ----------------------------------------------------------- | ------------------------------------------------------------ | | spatial relation | specify how one entity is located in relation to others | Gap spacer material is then deposited on the film knife-edge | | part-of | the ownership between two entities | a magnetic head has a magnetoresistive element | | causative relation | one entity operates as a cause of the other entity | Pressure pad carried another arm of spring urges film into contact with head | | operation | specify the relation between an activity and its object | Heat treatment improves the (100) orientation | | made-of | one entity is the material for making the other entity | The thin film head includes a substrate of electrically insulative material | | instance-of | the relation between a class and its instance | At least one of the magnetic layer is a free layer | | attribution | one entity is an attribution of the other entity | The thin film has very high heat resistance of remaining stable at 700.degree | | generating | one entity generates another entity | Buffer layer resistor create impedance that noise introduced to head from disk of drive | | purpose | relation between reason/result | conductor is utilized for producing writing flux in magnetic yoke | | in-manner-of | do something in certain way | The linear array is angled at a skew angle | | alias | one entity is also known under another entity’s name | The bias structure includes an antiferromagnetic layer AFM | | formation | an entity acts as a role of the other entity | Windings are joined at end to form center tapped winding | | comparison | compare one entity to the other | First end is closer to recording media use than second end | | measurement | one entity acts as a way to measure the other entity | This provides a relative permeance of at least 1000 | | other | not belongs to the above types | Then, MR resistance estimate during polishing step is calculated from S value and K value | There are 1010 patent abstracts with 3,986 sentences in this corpus . We use a web-based annotation tool named Brat[2] for data labeling, and the annotated data is saved in '.ann' format. The benefit of 'ann' is that you can display and manipulate the annotated data once the TFH_Annotated_Dataset.zip is unzipped under corresponding repository of Brat. TFH_Annotated_Dataset contains 22,833 entity mentions and 17,412 semantic relation mentions. With TFH_Annotated_Dataset, we run two tasks of information extraction including named entity recognition with BiLSTM-CRF[3] and semantic relation extractionand with BiGRU-2ATTENTION[4]. For improving semantic representation of patent language, the word embeddings are trained with the abstract of 46,302 patents regarding magnetic head in hard disk drive, which turn out to improve the performance of named entity recognition by 0.3% and semantic relation extraction by about 2% in weighted average F1, compared to GloVe and the patent word embedding provided by Risch et al[5]. For named entity recognition, the weighted-average precision, recall, F1-value of BiLSTM-CRF on entity-level for the test set are 78.5%, 78.0%, and 78.2%, respectively. Although such performance is acceptable, it is still lower than its performance on general-purpose dataset by more than 10% in F1-value. The main reason is the limited amount of labeled dataset. The precision, recall, and F1-value for each type of entity is shown in Fig. 4. As to relation extraction, the weighted-average precision, recall, F1-value of BiGRU-2ATTENTION for the test set are 89.7%, 87.9%, and 88.6% with no_edge relations, and 32.3%, 41.5%, 36.3% without no_edge relations. ## Academic citing Chen, L., Xu, S*., Zhu, L. et al. A deep learning based method for extracting semantic information from patent documents. Scientometrics 125, 289–312 (2020). https://doi.org/10.1007/s11192-020-03634-y ## Paper link https://link.springer.com/article/10.1007/s11192-020-03634-y ## REFERENCE [1] Pérez-Pérez, M., Pérez-Rodríguez, G., Vazquez, M., Fdez-Riverola, F., Oyarzabal, J., Oyarzabal, J., Valencia,A., Lourenço, A., & Krallinger, M. (2017). Evaluation of chemical and gene/protein entity recognition systems at BioCreative V.5: The CEMP and GPRO patents tracks. In Proceedings of the Bio-Creative V.5 challenge evaluation workshop, pp. 11–18. [2] Stenetorp, P., Pyysalo, S., Topić, G., Ohta, T., Ananiadou, S., & Tsujii, J. I. (2012). BRAT: a web-based tool for NLP-assisted text annotation. In Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics (pp. 102-107) [3] Huang, Z., Xu, W., &Yu, K. (2015). Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint arXiv:1508.01991 [4] Han,X., Gao,T., Yao,Y., Ye,D., Liu,Z., Sun, M.(2019). OpenNRE: An Open and Extensible Toolkit for Neural Relation Extraction. arXiv preprint arXiv: 1301.3781 [5] Risch, J., & Krestel, R. (2019). Domain-specific word embeddings for patent classification. Data Technologies and Applications, 53(1), 108–122.

출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.