MLRIP: Pre-training a military language representation model with informative factual knowledge and professional knowledge base
원문 초록
Incorporating prior knowledge into pre-trained language models has proven to be effective for knowledge-driven NLP tasks, such as entity typing and relation extraction. Current pre-training procedures usually inject external knowledge into models by using knowledge masking, knowledge fusion and knowledge replacement. However, factual information contained in the input sentences have not been fully mined, and the external knowledge for injecting have not been strictly checked. As a result, the context information cannot be fully exploited and extra noise will be introduced or the amount of knowledge injected is limited. To address these issues, we propose MLRIP, which modifies the knowledge masking strategies proposed by ERNIE-Baidu, and introduce a two-stage entity replacement strategy. Extensive experiments with comprehensive analyses illustrate the superiority of MLRIP over BERT-based models in military knowledge-driven NLP tasks.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
REXEL: An End-to-end Model for Document-Level Relation Extraction and Entity Linking
GeoLM: Empowering Language Models for Geospatially Grounded Language Understanding
Chaining Event Spans for Temporal Relation Grounding
Comparative Analysis of AI Agent Architectures for Entity Relationship Classification
Towards a More Generalized Approach in Open Relation Extraction
Towards Rehearsal-Free Continual Relation Extraction: Capturing Within-Task Variance with Adaptive Prompting
토론
이 논문과 연결된 포럼 글이 아직 없습니다.